From: Garot Conklin Date: August 22 2012 11:51pm Subject: Re: Machine Learning List-Archive: http://lists.mysql.com/mysql/228056 Message-Id: <1345679509.53279.YahooMailNeo@web140506.mail.bf1.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="1594581660-2086727025-1345679509=:53279" --1594581660-2086727025-1345679509=:53279 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable I am going to work up a visual diagram for this to better... visualize (LOL= ) the lay of the land. I will forward it when i have it completed then I ho= pe we can continue the architectural discussion here!=C2=A0 Thanks=0A=0A-Ga= rot=0A=0A=C2=A0=0Agarotconklin@stripped=0A=0A=0A__________________________= ______=0A From: Garot Conklin =0ATo: Garot Conklin = ; "webmaster@stripped" ; = 'Martin Gainty' ; "mysql@stripped" =0ASent: Tuesday, August 21, 2012 4:40 PM=0ASubject: Re: Machin= e Learning=0A =0Ato elaborate:=0A=0AI would almost rather qualify from the = OS perspective what will work in advance, but take into consideration any f= ailing condition.=C2=A0 I may be getting a bit too specific at this junctur= e however as I am already apply this logic to issues I see in MY environmen= t rather than total ambiguity, which is a best position here.=C2=A0 =0A=0AI= am ultimately expecting the learning algorithm to create new and excitingl= y effective ways of stringing commands together to solve issues, this being= a notable aspect in and of itself, then creating some centralization based= on what the machine determines as a best solution. =0A=0ASeems a relative = "snapshot of the system" pre and post execution would be favorable as well = so long as resources are not taxed as a result.=C2=A0 This discussion is le= ading me to believe that this will be a multifaceted db solution without a = doubt comprised of multiple layers of abstraction ultimately rolling up int= o a single master cluster for all the higher level analytics and internal t= esting cycles to then roll out the new primary protocols, which may themsel= ves take place in a sub-instance of each possible scenario.=C2=A0 =0A=0AI h= ave also though of populating a db or multiple db's with actual text=C2=A0 = in the form of .pdf's of actual technical manuals... not certain the benefi= t of the written language abstraction there but the thought process extract= ion might be an interesting end indeed.=0A=0A=0A=C2=A0=0Agarotconklin@yahoo= .com=0A=0A=0A________________________________=0AFrom: Garot Conklin =0ATo: "webmaster@stripped" ; 'M= artin Gainty' ; "mysql@stripped" =0ASent: Tuesday, August 21, 2012 4:09 PM=0ASubject: Re: Machine = Learning=0A=0AYES!=0A=0A=C2=A0=0Agarotconklin@stripped=0A=0A=0A___________= _____________________=0AFrom: "webmaster@stripped" =0ATo: 'Garot Conklin' ; 'Martin Gainty' ; mysql@stripped =0ASent: Tuesday, August 21, 2012 3:37= PM=0ASubject: RE: Machine Learning=0A=0A=0AAh,=0A=C2=A0=0AGetting clearer = and clearer.=0A=C2=A0=0ASo these =E2=80=98nodes=E2=80=99 could =E2=80=98lea= rn=E2=80=99 and =E2=80=98teach=E2=80=99 at the same time =E2=80=93 right ? = For instance, N1 runs a command in =E2=80=98domain=E2=80=99 D20 which it su= ccessful =E2=80=93 it send information to node N20 that is the authority on= domain D20 and N20 records it as success; N5 runs a command in domain D20 = which goes wrong, and sends info to N20 and N20 records failure and sends a= correct call to N5. Is this what you have in mind ?=0A=C2=A0=0AThere are a= lot of variables to be considered, for instance:=0A1)=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 Command A version 1 can run very happily on operating system vers= ion 2, but fails in OS version 1=0A2)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Command= A version 1 can run happily on OS version 2 on a 2G RAM, but fail on OS ve= rsion 2 on 1G RAM=0A3)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 etc=0A=C2=A0=0AI think= the DB design issues will become straightforward once the model is quite c= lear.=0A=C2=A0=0AJustin=0A=C2=A0=0AFrom:Garot Conklin [mailto:garotconklin@= yahoo.com] =0ASent: 21 August 2012 17:14=0ATo: webmaster@stripped; 'Mart= in Gainty'; mysql@stripped=0ASubject: Re: Machine Learning=0A=C2=A0= =0A1)Refer to it for =E2=80=98knowledge=E2=80=99 (for instance, of what the= latest=0Aversion of a command is)=0A=C2=A0=C2=A0=C2=A0 It would most likel= y end up being "central" in this sense:=0A=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0= =C2=A0 A distributed collection of systems; i.e. (possible defined in roles= )=0A=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 DB's=0A=C2=A0= =C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 FE's=0A=C2=A0=C2=A0=C2= =A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 REPL's=0A=C2=A0=C2=A0=C2=A0 =C2= =A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 Cache's=0A=C2=A0=C2=A0=C2=A0 =C2=A0=C2= =A0=C2=A0 Each DB would have its own collection of remediation's that would= then be indexed to populate a central db for trending/correlation etc...= =0A=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 The "Collective" itself would func= tion as a single conceptual implementation. A VIP for example might be asso= ciated with a specific role, say Web FE's and remediate only/all of them, b= ut only have some relative access to the core db of say the Network from a = Primary index perspective to make/draw associations/conclusions to issues a= t hand.=0A=C2=A0=0A=0A2)Send their =E2=80=98knowledge=E2=80=99 (for instanc= e, of the latest command=0Aversions) to it for storage and distribution to = others=0A=C2=A0=C2=A0=C2=A0Rather than "latest command versions" I envision= this to be more encapsulated as "latest successful invocation of the comma= nd string" and it inverse as well (to trend/metric-ize the failures thus le= nding to perpetual optimization).=0A=C2=A0=0AI like how this is fleshing ou= t... This is helping me to define what I am really trying to accomplish. Th= anks very much for everyone responding here, this is wonderful, please keep= this going...=0A=C2=A0=0Agarotconklin@stripped=0A=0A_____________________= ___________=0A=0AFrom:"webmaster@stripped" =0ATo:= 'Martin Gainty' ; garotconklin@stripped; mysql@lists= .mysql.com =0ASent: Tuesday, August 21, 2012 11:19 AM=0ASubject: RE: Machin= e Learning=0A=0AHi Garot,=0A=0A=0A=0AOk, the concept is getting clearer, bu= t let=E2=80=99s bring this down to earth a=0Alittle bit more. I love DB des= ign and problem-solving and am quite curious=0Aabout this.=0A=0A=0A=0AIs th= e idea that you have a central computer (not HAL J) somewhere so that=0Aoth= er computers can:=0A=0A1)=C2=A0 =C2=A0 =C2=A0 Refer to it for =E2=80=98know= ledge=E2=80=99 (for instance, of what the latest=0Aversion of a command is)= =0A=0A2)=C2=A0 =C2=A0 =C2=A0 Send their =E2=80=98knowledge=E2=80=99 (for in= stance, of the latest command=0Aversions) to it for storage and distributio= n to others=0A=0A=0A=0AIf this is the model, then the knowledge base can bu= ild up organically over=0Atime =E2=80=93 I think. Or is this too simplistic= ?=0A=0A=0A=0AThanks,=0A=0AJustin=0A=0A=0A=0AFrom: Martin Gainty [mailto:mg= ainty@stripped] =0ASent: 21 August 2012 00:25=0ATo: garotconklin@stripped= om; webmaster@stripped; mysql@stripped=0ASubject: RE: Machine Lea= rning=0A=0A=0A=0AWhen I hear 'AI' I always imagine theres a HAL 9001 behind= the scenes that=0Ais running the show constantly admonishing its creator t= o "take another=0Astress pill"=0A=0ASounds like a fun project=0A=0AKeep us = apprised,=0AMartin Gainty =0A______________________________________________= =0AVerzicht und Vertraulichkeitanmerkung/Note de d=C3=A9ni et de confident= ialit=C3=A9=0A=0A=0ADiese Nachricht ist vertraulich. Sollten Sie nicht der = vorgesehene=0AEmpfaenger sein, so bitten wir hoeflich um eine Mitteilung. J= ede unbefugte=0AWeiterleitung oder Fertigung einer Kopie ist unzulaessig. D= iese Nachricht=0Adient lediglich dem Austausch von Informationen und entfal= tet keine=0Arechtliche Bindungswirkung. Aufgrund der leichten Manipulierbar= keit von=0AE-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.= =0A=0ACe message est confidentiel et peut =C3=AAtre privil=C3=A9gi=C3=A9. S= i vous n'=C3=AAtes pas le=0Adestinataire pr=C3=A9vu, nous te demandons avec= bont=C3=A9 que pour satisfaire=0Ainformez l'exp=C3=A9diteur. N'importe que= lle diffusion non autoris=C3=A9e ou la copie=0Ade ceci est interdite. Ce me= ssage sert =C3=A0 l'information seulement et n'aura=0Apas n'importe quel ef= fet l=C3=A9galement obligatoire. =C3=89tant donn=C3=A9 que les email=0Apeuv= ent facilement =C3=AAtre sujets =C3=A0 la manipulation, nous ne pouvons acc= epter=0Aaucune responsabilit=C3=A9 pour le contenu fourni.=0A=0A=0A=0A=0A= =0A=C2=A0 _____=C2=A0 =0A=0ADate: Mon, 20 Aug 2012 13:50:04 -0700=0AFrom: g= arotconklin@stripped=0ASubject: Re: Machine Learning=0ATo: webmaster@lisol= .co.uk; mgainty@stripped; mysql@stripped=0A=0AYa the idea is not = anything new, but must be apparently quit difficult or=0Anot a priority as = I have yet to find it already implemented anywhere... Far=0Abe it from me t= o not make some attempt here anyway...=0A=0A=0A=0AI am creating a fully aut= omated framework from which a distributed=0Ainfrastructure can be maintaine= d.=C2=A0 I have been writing automation=0Ascripts/code for some time now an= d the logical progression is to embark on a=0Afull concept of systems healt= h auto remediation.=C2=A0 I have numerous=0A"monitoring" solutions under my= control however none that properly (in my=0Aopinion) implements any real l= earning algorithms from which to draw even a=0Aminimalists view of automati= on.=C2=A0 I like mySQL therefor began thinking about=0Acreating the aspects= (lobes) of the "brain" as a relational database(s).=C2=A0 So=0Athis is onl= y one facet of what I am trying to do, however leveraging a full=0Acommand = set of shell utilities/commands/programs seemed to be a good=0Astarting poi= nt before I get into the "hard" stuff ! =0A=0A=0A=0A-Garot=0A=0A=0A=0Agarot= conklin@stripped=0A=0A=C2=A0 _____=C2=A0 =0A=0AFrom: "webmaster@stripped= k" =0ATo: 'Martin Gainty' ; gar= otconklin@stripped;=0Amysql@stripped =0ASent: Monday, August 20, 20= 12 3:55 PM=0ASubject: RE: Machine Learning=0A=0A=0AHi Garot,=0A=0A=0A=0AYou= 'll have to elaborate some more ... I understand you may want to protect=0A= the idea as well, so if you can narrow it down to some technical specifics= =0Athen it'll help.=0A=0A=0A=0AWhat is the objective of this system, for in= stance ?=0A=0A=0A=0AThanks,=0A=0AJustin=0A=0A=0A=0AFrom: Martin Gainty [mai= lto:mgainty@stripped] =0ASent: 20 August 2012 19:23=0ATo: garotconklin@y= ahoo.com; webmaster@stripped; mysql@stripped=0ASubject: RE: Machi= ne Learning=0A=0A=0A=0A=0A=0AFrom: garotconklin@stripped=0ASubject: Re: Ma= chine Learning=0ATo: webmaster@stripped; mgainty@stripped; mysql@list= s.mysql.com=0A=0A=0AMy initial thought was to propagate the db with everyth= ing and allow the=0Aalgorithm to then begin to determin trends/patterns=0AM= G>which trends or patterns will you be modelling?=0A=0Aand begin either an = indexing methodology=0AMG>which indexes are you considering: Unique index, = primary index or foreign=0Aindex?=0A=0Aadditional table/db creation process= or both to further optimize the calls=0Abeing made=0AMG>optimize based on = execution time or diskspace allocated, EliminatingFTS=0Aor some other crite= ria?=0AMG>https://dev.mysql.com/doc/refman/5.5/en/optimization.html=0A=0Aan= d build in some internal levels of redundancy.=0AMG>what about replication= =0AMG>http://dev.mysql.com/doc/refman/5.5/en/replication.html=0A=0AI am act= ually approaching this with some degree of biological conception in=0Athe m= ultipathing within our own brains however until I have something up and=0Ar= unning under some substantial load however I may not get a complete=0Apictu= re. =0A=0AThanks,=0A=0AGarot=0A=0A=0AInteresting=0AMartin=0A=0A=C2=A0 _____= =C2=A0 =0A=0AFrom: webmaster@stripped ; =0ATo: 'G= arot Conklin' ; 'Martin Gainty'=0A; ; =0ASubject: RE: Machine Learning =0ASent: Mo= n, Aug 20, 2012 7:13:25 AM =0A=0A=0AHi Garot,=0A=0AThis sounds an interesti= ng idea.=0A=0AAre you looking to store all known commands and their options= or are you=0Alooking for a 'formula' for calling any unix command ?=0A=0AT= he reason for my question is that, at the end of the day, a unix command is= =0Ajust a program that is run in the operating system. Each program comes w= ith=0Aits own options and acceptable inputs. I don't know if there is a rul= e or=0Aconvention for structuring these commands.=0A=0AAre you then looking= to build a system that 'knows' all commands and 'how=0Ato' call them ?=0A= =0AThanks,=0AJustin=0A=0A-----Original Message-----=0AFrom: Garot Conklin [= mailto:garotconklin@stripped] =0ASent: 20 August 2012 03:39=0ATo: Martin G= ainty; mysql@stripped=0ASubject: Re: Machine Learning=0A=0AThe initi= al goal is to provide a working framework from which to call all=0AUNIX she= ll command combinations as the underlying storage mechanism for a=0Amachine= learning algorithm.=C2=A0 I would like to build a completely self aware=0A= instantiation that will maintain itself on all levels... I postulate that= =0Athe first place to start would be in determining a method for maintainin= g=0Aall possible remediation combinations including the unknown to eventual= ly be=0Alearned from and populate new knowledge into the database.=C2=A0 Th= ank you for=0Athe reply,=0A=0AGarot --1594581660-2086727025-1345679509=:53279--