List:General Discussion« Previous MessageNext Message »
From:Garot Conklin Date:August 22 2012 11:51pm
Subject:Re: Machine Learning
View as plain text  
I am going to work up a visual diagram for this to better... visualize (LOL) the lay of
the land. I will forward it when i have it completed then I hope we can continue the
architectural discussion here!  Thanks

-Garot

 
garotconklin@stripped


________________________________
 From: Garot Conklin <garotconklin@stripped>
To: Garot Conklin <garotconklin@stripped>; "webmaster@stripped"
<webmaster@stripped>; 'Martin Gainty' <mgainty@stripped>;
"mysql@stripped" <mysql@stripped> 
Sent: Tuesday, August 21, 2012 4:40 PM
Subject: Re: Machine Learning
 
to elaborate:

I would almost rather qualify from the OS perspective what will work in advance, but take
into consideration any failing condition.  I may be getting a bit too specific at
this juncture however as I am already apply this logic to issues I see in MY environment
rather than total ambiguity, which is a best position here.  

I am ultimately expecting the learning algorithm to create new and excitingly effective
ways of stringing commands together to solve issues, this being a notable aspect in and
of itself, then creating some centralization based on what the machine determines as a
best solution. 

Seems a relative "snapshot of the system" pre and post execution would be favorable as
well so long as resources are not taxed as a result.  This discussion is leading me
to believe that this will be a multifaceted db solution without a doubt comprised of
multiple layers of abstraction ultimately rolling up into a single master cluster for all
the higher level analytics and internal testing cycles to then roll out the new primary
protocols, which may themselves take place in a sub-instance of each possible
scenario.  

I have also though of populating a db or multiple db's with actual text  in the form
of .pdf's of actual technical manuals... not certain the benefit of the written language
abstraction there but the thought process extraction might be an interesting end indeed.


 
garotconklin@stripped


________________________________
From: Garot Conklin <garotconklin@stripped>
To: "webmaster@stripped" <webmaster@stripped>; 'Martin Gainty'
<mgainty@stripped>; "mysql@stripped" <mysql@stripped> 
Sent: Tuesday, August 21, 2012 4:09 PM
Subject: Re: Machine Learning

YES!

 
garotconklin@stripped


________________________________
From: "webmaster@stripped" <webmaster@stripped>
To: 'Garot Conklin' <garotconklin@stripped>; 'Martin Gainty'
<mgainty@stripped>; mysql@stripped 
Sent: Tuesday, August 21, 2012 3:37 PM
Subject: RE: Machine Learning


Ah,
 
Getting clearer and clearer.
 
So these ‘nodes’ could ‘learn’ and ‘teach’ at the
same time – right ? For instance, N1 runs a command in ‘domain’ D20
which it successful – it send information to node N20 that is the authority on
domain D20 and N20 records it as success; N5 runs a command in domain D20 which goes
wrong, and sends info to N20 and N20 records failure and sends a correct call to N5. Is
this what you have in mind ?
 
There are a lot of variables to be considered, for instance:
1)      Command A version 1 can run very happily on operating
system version 2, but fails in OS version 1
2)      Command A version 1 can run happily on OS version 2 on a
2G RAM, but fail on OS version 2 on 1G RAM
3)      etc
 
I think the DB design issues will become straightforward once the model is quite clear.
 
Justin
 
From:Garot Conklin [mailto:garotconklin@stripped] 
Sent: 21 August 2012 17:14
To: webmaster@stripped; 'Martin Gainty'; mysql@stripped
Subject: Re: Machine Learning
 
1)Refer to it for ‘knowledge’ (for instance, of what the latest
version of a command is)
    It would most likely end up being "central" in this sense:
        A distributed collection of systems; i.e. (possible
defined in roles)
            DB's
            FE's
            REPL's
            Cache's
        Each DB would have its own collection of
remediation's that would then be indexed to populate a central db for
trending/correlation etc...
        The "Collective" itself would function as a single
conceptual implementation. A VIP for example might be associated with a specific role,
say Web FE's and remediate only/all of them, but only have some relative access to the
core db of say the Network from a Primary index perspective to make/draw
associations/conclusions to issues at hand.
 

2)Send their ‘knowledge’ (for instance, of the latest command
versions) to it for storage and distribution to others
   Rather than "latest command versions" I envision this to be more
encapsulated as "latest successful invocation of the command string" and it inverse as
well (to trend/metric-ize the failures thus lending to perpetual optimization).
 
I like how this is fleshing out... This is helping me to define what I am really trying
to accomplish. Thanks very much for everyone responding here, this is wonderful, please
keep this going...
 
garotconklin@stripped

________________________________

From:"webmaster@stripped" <webmaster@stripped>
To: 'Martin Gainty' <mgainty@stripped>; garotconklin@stripped;
mysql@stripped 
Sent: Tuesday, August 21, 2012 11:19 AM
Subject: RE: Machine Learning

Hi Garot,



Ok, the concept is getting clearer, but let’s bring this down to earth a
little bit more. I love DB design and problem-solving and am quite curious
about this.



Is the idea that you have a central computer (not HAL J) somewhere so that
other computers can:

1)      Refer to it for ‘knowledge’ (for instance, of what the
latest
version of a command is)

2)      Send their ‘knowledge’ (for instance, of the latest
command
versions) to it for storage and distribution to others



If this is the model, then the knowledge base can build up organically over
time – I think. Or is this too simplistic ?



Thanks,

Justin



From: Martin Gainty [mailto:mgainty@stripped] 
Sent: 21 August 2012 00:25
To: garotconklin@stripped; webmaster@stripped; mysql@stripped
Subject: RE: Machine Learning



When I hear 'AI' I always imagine theres a HAL 9001 behind the scenes that
is running the show constantly admonishing its creator to "take another
stress pill"

Sounds like a fun project

Keep us apprised,
Martin Gainty 
______________________________________________ 
Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité


Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene
Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte
Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht
dient lediglich dem Austausch von Informationen und entfaltet keine
rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von
E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.

Ce message est confidentiel et peut être privilégié. Si vous
n'êtes pas le
destinataire prévu, nous te demandons avec bonté que pour satisfaire
informez l'expéditeur. N'importe quelle diffusion non autorisée ou la copie
de ceci est interdite. Ce message sert à l'information seulement et n'aura
pas n'importe quel effet légalement obligatoire. Étant donné que les
email
peuvent facilement être sujets à la manipulation, nous ne pouvons accepter
aucune responsabilité pour le contenu fourni.





  _____  

Date: Mon, 20 Aug 2012 13:50:04 -0700
From: garotconklin@stripped
Subject: Re: Machine Learning
To: webmaster@stripped; mgainty@stripped; mysql@stripped

Ya the idea is not anything new, but must be apparently quit difficult or
not a priority as I have yet to find it already implemented anywhere... Far
be it from me to not make some attempt here anyway...



I am creating a fully automated framework from which a distributed
infrastructure can be maintained.  I have been writing automation
scripts/code for some time now and the logical progression is to embark on a
full concept of systems health auto remediation.  I have numerous
"monitoring" solutions under my control however none that properly (in my
opinion) implements any real learning algorithms from which to draw even a
minimalists view of automation.  I like mySQL therefor began thinking about
creating the aspects (lobes) of the "brain" as a relational database(s).  So
this is only one facet of what I am trying to do, however leveraging a full
command set of shell utilities/commands/programs seemed to be a good
starting point before I get into the "hard" stuff ! 



-Garot



garotconklin@stripped

  _____  

From: "webmaster@stripped" <webmaster@stripped>
To: 'Martin Gainty' <mgainty@stripped>; garotconklin@stripped;
mysql@stripped 
Sent: Monday, August 20, 2012 3:55 PM
Subject: RE: Machine Learning


Hi Garot,



You'll have to elaborate some more ... I understand you may want to protect
the idea as well, so if you can narrow it down to some technical specifics
then it'll help.



What is the objective of this system, for instance ?



Thanks,

Justin



From: Martin Gainty [mailto:mgainty@stripped] 
Sent: 20 August 2012 19:23
To: garotconklin@stripped; webmaster@stripped; mysql@stripped
Subject: RE: Machine Learning





From: garotconklin@stripped
Subject: Re: Machine Learning
To: webmaster@stripped; mgainty@stripped; mysql@stripped


My initial thought was to propagate the db with everything and allow the
algorithm to then begin to determin trends/patterns
MG>which trends or patterns will you be modelling?

and begin either an indexing methodology
MG>which indexes are you considering: Unique index, primary index or foreign
index?

additional table/db creation process or both to further optimize the calls
being made
MG>optimize based on execution time or diskspace allocated, EliminatingFTS
or some other criteria?
MG>https://dev.mysql.com/doc/refman/5.5/en/optimization.html

and build in some internal levels of redundancy.
MG>what about replication
MG>http://dev.mysql.com/doc/refman/5.5/en/replication.html

I am actually approaching this with some degree of biological conception in
the multipathing within our own brains however until I have something up and
running under some substantial load however I may not get a complete
picture. 

Thanks,

Garot


Interesting
Martin

  _____  

From: webmaster@stripped <webmaster@stripped>; 
To: 'Garot Conklin' <garotconklin@stripped>; 'Martin Gainty'
<mgainty@stripped>; <mysql@stripped>; 
Subject: RE: Machine Learning 
Sent: Mon, Aug 20, 2012 7:13:25 AM 


Hi Garot,

This sounds an interesting idea.

Are you looking to store all known commands and their options or are you
looking for a 'formula' for calling any unix command ?

The reason for my question is that, at the end of the day, a unix command is
just a program that is run in the operating system. Each program comes with
its own options and acceptable inputs. I don't know if there is a rule or
convention for structuring these commands.

Are you then looking to build a system that 'knows' all commands and 'how
to' call them ?

Thanks,
Justin

-----Original Message-----
From: Garot Conklin [mailto:garotconklin@stripped] 
Sent: 20 August 2012 03:39
To: Martin Gainty; mysql@stripped
Subject: Re: Machine Learning

The initial goal is to provide a working framework from which to call all
UNIX shell command combinations as the underlying storage mechanism for a
machine learning algorithm.  I would like to build a completely self aware
instantiation that will maintain itself on all levels... I postulate that
the first place to start would be in determining a method for maintaining
all possible remediation combinations including the unknown to eventually be
learned from and populate new knowledge into the database.  Thank you for
the reply,

Garot
Thread
Machine LearningGarot Conklin19 Aug
  • RE: Machine LearningMartin Gainty19 Aug
Re: Machine LearningGarot Conklin20 Aug
  • RE: Machine Learningwebmaster20 Aug
Re: Machine LearningGarot Conklin20 Aug
  • RE: Machine LearningMartin Gainty20 Aug
    • Re: Machine LearningGarot Conklin20 Aug
    • RE: Machine Learningwebmaster20 Aug
      • Re: Machine LearningGarot Conklin20 Aug
        • RE: Machine LearningMartin Gainty20 Aug
          • Re: Machine LearningMichael21 Aug
            • RE: Machine LearningMartin Gainty21 Aug
              • Re: Machine LearningGarot Conklin21 Aug
          • RE: Machine Learningwebmaster21 Aug
            • Re: Machine LearningGarot Conklin21 Aug
              • RE: Machine Learningwebmaster21 Aug
                • RE: Machine LearningMartin Gainty21 Aug
                • RE: Machine LearningMartin Gainty21 Aug
                • RE: Machine LearningMartin Gainty21 Aug
                • Re: Machine LearningGarot Conklin21 Aug
                  • Re: Machine LearningGarot Conklin21 Aug
                    • Re: Machine LearningGarot Conklin22 Aug
                      • Re: Machine Learningshawn wilson23 Aug
RE: Machine LearningMartin Gainty23 Aug
  • Re: Machine LearningGarot Conklin23 Aug
    • RE: Machine Learningwebmaster24 Aug
      • RE: Machine LearningMartin Gainty24 Aug
      • Re: Machine LearningGarot Conklin24 Aug