From: Martin Gainty Date: August 21 2012 8:04pm Subject: RE: Machine Learning List-Archive: http://lists.mysql.com/mysql/228038 Message-Id: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="_7963c2e6-690b-4b4a-ba8e-a33762db7b5c_" --_7963c2e6-690b-4b4a-ba8e-a33762db7b5c_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Im thinking any of the JSR-286 Portal Management Systems can *probably* han= dle initial authentication and authentication-token access to all of the re= sources via token passing http://en.wikipedia.org/wiki/List_of_enterprise_portal_vendors http://portals.apache.org/jetspeed-2/ As far event models you might want to think of Observer Pattern specificall= y when you want one or more Nodes to observe or be notified of Change in Su= bject or delta-property http://userpages.umbc.edu/~tarr/dp/lectures/Observer.pdf Here is a concrete Java example of Observer pattern in Java http://www.vogella.com/articles/DesignPatternObserver/article.html As far as HW Requirements..the only requirement I see is for a MultiProcess= or with some *kind of* LoadBalancer to arbitrate Load between the 1..n Node= s WDYT? Martin=20 ______________________________________________=20 Verzicht und Vertraulichkeitanmerkung/Note de d=E9ni et de confidentialit= =E9 Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaeng= er sein=2C so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiter= leitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht dient l= ediglich dem Austausch von Informationen und entfaltet keine rechtliche Bin= dungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen w= ir keine Haftung fuer den Inhalt uebernehmen. Ce message est confidentiel et peut =EAtre privil=E9gi=E9. Si vous n'=EAtes= pas le destinataire pr=E9vu=2C nous te demandons avec bont=E9 que pour sat= isfaire informez l'exp=E9diteur. N'importe quelle diffusion non autoris=E9e= ou la copie de ceci est interdite. Ce message sert =E0 l'information seule= ment et n'aura pas n'importe quel effet l=E9galement obligatoire. =C9tant d= onn=E9 que les email peuvent facilement =EAtre sujets =E0 la manipulation= =2C nous ne pouvons accepter aucune responsabilit=E9 pour le contenu fourni= . > From: webmaster@stripped > To: garotconklin@stripped=3B mgainty@stripped=3B mysql@stripped= m > Subject: RE: Machine Learning > Date: Tue=2C 21 Aug 2012 20:37:45 +0100 >=20 > Ah=2C >=20 > =20 >=20 > Getting clearer and clearer. >=20 > =20 >=20 > So these =91nodes=92 could =91learn=92 and =91teach=92 at the same time = =96 right ? For instance=2C N1 runs a command in =91domain=92 D20 which it = successful =96 it send information to node N20 that is the authority on dom= ain D20 and N20 records it as success=3B N5 runs a command in domain D20 wh= ich goes wrong=2C and sends info to N20 and N20 records failure and sends a= correct call to N5. Is this what you have in mind ? >=20 > =20 >=20 > There are a lot of variables to be considered=2C for instance: >=20 > 1) Command A version 1 can run very happily on operating system vers= ion 2=2C but fails in OS version 1 >=20 > 2) Command A version 1 can run happily on OS version 2 on a 2G RAM= =2C but fail on OS version 2 on 1G RAM >=20 > 3) etc >=20 > =20 >=20 > I think the DB design issues will become straightforward once the model i= s quite clear. >=20 > =20 >=20 > Justin >=20 > =20 >=20 > From: Garot Conklin [mailto:garotconklin@stripped]=20 > Sent: 21 August 2012 17:14 > To: webmaster@stripped=3B 'Martin Gainty'=3B mysql@stripped > Subject: Re: Machine Learning >=20 > =20 >=20 > 1)Refer to it for =91knowledge=92 (for instance=2C of what the latest > version of a command is) >=20 > It would most likely end up being "central" in this sense: >=20 > A distributed collection of systems=3B i.e. (possible defined in = roles) >=20 > DB's >=20 > FE's >=20 > REPL's >=20 > Cache's >=20 > Each DB would have its own collection of remediation's that would= then be indexed to populate a central db for trending/correlation etc... >=20 > The "Collective" itself would function as a single conceptual imp= lementation. A VIP for example might be associated with a specific role=2C = say Web FE's and remediate only/all of them=2C but only have some relative = access to the core db of say the Network from a Primary index perspective t= o make/draw associations/conclusions to issues at hand. >=20 > =20 >=20 >=20 > 2)Send their =91knowledge=92 (for instance=2C of the latest command > versions) to it for storage and distribution to others >=20 > Rather than "latest command versions" I envision this to be more enca= psulated as "latest successful invocation of the command string" and it inv= erse as well (to trend/metric-ize the failures thus lending to perpetual op= timization). >=20 > =20 >=20 > I like how this is fleshing out... This is helping me to define what I am= really trying to accomplish. Thanks very much for everyone responding here= =2C this is wonderful=2C please keep this going... >=20 > =20 >=20 > garotconklin@stripped >=20 > _____ =20 >=20 > From: "webmaster@stripped" > To: 'Martin Gainty' =3B garotconklin@stripped=3B my= sql@stripped=20 > Sent: Tuesday=2C August 21=2C 2012 11:19 AM > Subject: RE: Machine Learning >=20 >=20 > Hi Garot=2C >=20 >=20 >=20 > Ok=2C the concept is getting clearer=2C but let=92s bring this down to ea= rth a > little bit more. I love DB design and problem-solving and am quite curiou= s > about this. >=20 >=20 >=20 > Is the idea that you have a central computer (not HAL J) somewhere so tha= t > other computers can: >=20 > 1) Refer to it for =91knowledge=92 (for instance=2C of what the late= st > version of a command is) >=20 > 2) Send their =91knowledge=92 (for instance=2C of the latest command > versions) to it for storage and distribution to others >=20 >=20 >=20 > If this is the model=2C then the knowledge base can build up organically = over > time =96 I think. Or is this too simplistic ? >=20 >=20 >=20 > Thanks=2C >=20 > Justin >=20 >=20 >=20 > From: Martin Gainty [mailto:mgainty@stripped]=20 > Sent: 21 August 2012 00:25 > To: garotconklin@stripped=3B webmaster@stripped=3B mysql@stripped.= com > Subject: RE: Machine Learning >=20 >=20 >=20 > When I hear 'AI' I always imagine theres a HAL 9001 behind the scenes tha= t > is running the show constantly admonishing its creator to "take another > stress pill" >=20 > Sounds like a fun project >=20 > Keep us apprised=2C > Martin Gainty=20 > ______________________________________________=20 > Verzicht und Vertraulichkeitanmerkung/Note de d=E9ni et de confidentialit= =E9 >=20 >=20 > Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene > Empfaenger sein=2C so bitten wir hoeflich um eine Mitteilung. Jede unbefu= gte > Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht > dient lediglich dem Austausch von Informationen und entfaltet keine > rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von > E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen. >=20 > Ce message est confidentiel et peut =EAtre privil=E9gi=E9. Si vous n'=EAt= es pas le > destinataire pr=E9vu=2C nous te demandons avec bont=E9 que pour satisfair= e > informez l'exp=E9diteur. N'importe quelle diffusion non autoris=E9e ou la= copie > de ceci est interdite. Ce message sert =E0 l'information seulement et n'a= ura > pas n'importe quel effet l=E9galement obligatoire. =C9tant donn=E9 que le= s email > peuvent facilement =EAtre sujets =E0 la manipulation=2C nous ne pouvons a= ccepter > aucune responsabilit=E9 pour le contenu fourni. >=20 >=20 >=20 >=20 >=20 > _____ =20 >=20 > Date: Mon=2C 20 Aug 2012 13:50:04 -0700 > From: garotconklin@stripped > Subject: Re: Machine Learning > To: webmaster@stripped=3B mgainty@stripped=3B mysql@stripped >=20 > Ya the idea is not anything new=2C but must be apparently quit difficult = or > not a priority as I have yet to find it already implemented anywhere... F= ar > be it from me to not make some attempt here anyway... >=20 >=20 >=20 > I am creating a fully automated framework from which a distributed > infrastructure can be maintained. I have been writing automation > scripts/code for some time now and the logical progression is to embark o= n a > full concept of systems health auto remediation. I have numerous > "monitoring" solutions under my control however none that properly (in my > opinion) implements any real learning algorithms from which to draw even = a > minimalists view of automation. I like mySQL therefor began thinking abo= ut > creating the aspects (lobes) of the "brain" as a relational database(s). = So > this is only one facet of what I am trying to do=2C however leveraging a = full > command set of shell utilities/commands/programs seemed to be a good > starting point before I get into the "hard" stuff !=20 >=20 >=20 >=20 > -Garot >=20 >=20 >=20 > garotconklin@stripped >=20 > _____ =20 >=20 > From: "webmaster@stripped" > To: 'Martin Gainty' =3B garotconklin@stripped=3B > mysql@stripped=20 > Sent: Monday=2C August 20=2C 2012 3:55 PM > Subject: RE: Machine Learning >=20 >=20 > Hi Garot=2C >=20 >=20 >=20 > You'll have to elaborate some more ... I understand you may want to prote= ct > the idea as well=2C so if you can narrow it down to some technical specif= ics > then it'll help. >=20 >=20 >=20 > What is the objective of this system=2C for instance ? >=20 >=20 >=20 > Thanks=2C >=20 > Justin >=20 >=20 >=20 > From: Martin Gainty [mailto:mgainty@stripped]=20 > Sent: 20 August 2012 19:23 > To: garotconklin@stripped=3B webmaster@stripped=3B mysql@stripped.= com > Subject: RE: Machine Learning >=20 >=20 >=20 >=20 >=20 > From: garotconklin@stripped > Subject: Re: Machine Learning > To: webmaster@stripped=3B mgainty@stripped=3B mysql@stripped >=20 >=20 > My initial thought was to propagate the db with everything and allow the > algorithm to then begin to determin trends/patterns > MG>which trends or patterns will you be modelling? >=20 > and begin either an indexing methodology > MG>which indexes are you considering: Unique index=2C primary index or fo= reign > index? >=20 > additional table/db creation process or both to further optimize the call= s > being made > MG>optimize based on execution time or diskspace allocated=2C Eliminating= FTS > or some other criteria? > MG>https://dev.mysql.com/doc/refman/5.5/en/optimization.html >=20 > and build in some internal levels of redundancy. > MG>what about replication > MG>http://dev.mysql.com/doc/refman/5.5/en/replication.html >=20 > I am actually approaching this with some degree of biological conception = in > the multipathing within our own brains however until I have something up = and > running under some substantial load however I may not get a complete > picture.=20 >=20 > Thanks=2C >=20 > Garot >=20 >=20 > Interesting > Martin >=20 > _____ =20 >=20 > From: webmaster@stripped =3B=20 > To: 'Garot Conklin' =3B 'Martin Gainty' > =3B =3B=20 > Subject: RE: Machine Learning=20 > Sent: Mon=2C Aug 20=2C 2012 7:13:25 AM=20 >=20 >=20 > Hi Garot=2C >=20 > This sounds an interesting idea. >=20 > Are you looking to store all known commands and their options or are you > looking for a 'formula' for calling any unix command ? >=20 > The reason for my question is that=2C at the end of the day=2C a unix com= mand is > just a program that is run in the operating system. Each program comes wi= th > its own options and acceptable inputs. I don't know if there is a rule or > convention for structuring these commands. >=20 > Are you then looking to build a system that 'knows' all commands and 'how > to' call them ? >=20 > Thanks=2C > Justin >=20 > -----Original Message----- > From: Garot Conklin [mailto:garotconklin@stripped]=20 > Sent: 20 August 2012 03:39 > To: Martin Gainty=3B mysql@stripped > Subject: Re: Machine Learning >=20 > The initial goal is to provide a working framework from which to call all > UNIX shell command combinations as the underlying storage mechanism for a > machine learning algorithm. I would like to build a completely self awar= e > instantiation that will maintain itself on all levels... I postulate that > the first place to start would be in determining a method for maintaining > all possible remediation combinations including the unknown to eventually= be > learned from and populate new knowledge into the database. Thank you for > the reply=2C >=20 > Garot >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 = --_7963c2e6-690b-4b4a-ba8e-a33762db7b5c_--