List:MySQL++« Previous MessageNext Message »
From:Rick Gutleber Date:October 27 2008 10:58pm
Subject:Re: Insert policy design question
View as plain text  
Drew M. wrote:
> The other code in query.h uses an overloaded << operator with a Query object
> on the left side. Your code uses an ostringstream, which is why the behavior
> is different.
If you look at the code in row.h, from which the SSQLS-generated classes 
are derived, for the equal_list and value_list templates, you will see 
that they are designed to be used with C++ streams in general.  There is 
nothing specific to Query as far as I understand.  Certainly there is no 
overloaded << operator in query.h.  I thought perhaps there was 
something specific to Query at first, but was unable to find any reason 
in the code why there should be.

Of course, this is all predicated on the fact that I may have missed or 
misunderstood something, as I am still learning MySQL++ and it's not an 
unsophisticated piece of code.

> Some constructive feedback for discussion/consideration: I suggest you make
> your code conform more to the same style as the other insert-ish methods in
> Query. 
You might wish to peruse the mailing list history which contains our 
entire discussion from earlier in October.  Your feedback is certainly 
appreciated, and perhaps you might have something else to add to or 
improve in the original design. 

The InsertPolicy concept was Warren's idea because he didn't want to 
limit the criteria used to determine how bulk INSERT statements are built.
The constraints are the key difference between the insertfrom( ) method 
and the regular insert( iter1, iter2 ) method.  The topic that caused 
the initial discussion for this whole thing is the MySQL "max packet 
size" setting which determines how big a single SQL statement sent to 
MySQL can be.  I had heretofore been "manually" constraining the size of 
the INSERT statement at the application level, which is something I 
think could be transparently handled at the library level, since a major 
selling point of MySQL++ is that it frees you from the tedium of 
building workaday SQL statements.  Warren agreed that this could be a 
useful addition and suggested a way I could implement it.  Frankly, it 
took me quite a while to understand exactly what he meant (assuming I 
actually have done so ;-)), and I believe the prototypical 
implementation I have provided (it's not intended to be complete), 
accurately expresses Warren's idea in a way I think is useful.

> To do this, you would actually remove the add() method and keep the
> query-writing logic in the query class. Rather than have your policy object
> construct the query, you would construct the query within the insertfrom()
> call and then iterate through the list. 

The problem with that is that there is then really nothing for the 
InsertPolicy to do.  If it doesn't maintain the actual query it cannot 
determine whether another object can be added.  Passing the query in to 
each canAdd( ) call is another possibility, but given that you can 
construct a quoted query using any C++ stream (according to my 
understanding) you might as well let the InsertPolicy handle it.  
Another possibility that just occurred to me is creating the query in 
the InsertPolicy allows for supporting autoincrement ID columns, which 
SSQLS does not otherwise do.

> I am also curious if you meant to
> add an entire insert statement each time the policy's add() method is
> called. 

I believe you are misinterpreting the code.  Each call to add( ) adds an 
additional value list to the INSERT statement being maintained by the 
InsertPolicy object.  It only creates a new INSERT statement immediately 
subsequent to a call to startQuery( ).

> It seems like you would keep adding to the same statement until the
> policy decided that you had iterated enough (or the end of the list).

The idea behind the insert policy object was that there are special 
constraints that might apply to creating a bulk INSERT statement.  One 
constraint is the MySQL "max packet size" parameter I described 
earlier.  Warren's idea for the InsertPolicy was that there could be 
other constraints that users might wish to impose themselves using 
hand-rolled InsertPolicies.  MySQL++ would provide the 3 or 4 "obvious" 
policy objects, but there would be nothing stopping a user from passing 
any object conforming to the InsertPolicy interface expressed in the 
insertfrom( ) template method.

> I think you should take the transaction calls out, similar to the other
> insert-ish methods. 

Again, this was Warren's suggestion, and I think it's not only a good 
idea, but absolutely necessary.  If you call insertfrom( ), unlike the 
other methods in mysqlpp::Query it is possible for the method to fail 
_partway_ through, i.e., some records are inserted, but not all.   It 
only makes sense that you use a transaction, because otherwise the 
result of a failed operation is indeterminate, which is always Very Bad.

> It should be up to the end-developer to decide whether
> or not to employ transactions, and using them in library code might cause
> unintended (and unavoidable) effects for end-developers.

Normally, I would agree, but for the reasons above, I don't think it's 
avoidable.  This method is, as with most parts of MySQL++, only one way 
to do a particular operation.  Given proper caveats in the 
documentation, the user can determine whether this method would cause a 
problem, and if so, there are plenty of other, albeit less convenient, 
ways to achieve the same end.

> I would suggest instead throwing an exception (if the query has exceptions
> enabled, that is). This is more in line with the rest of the Query class.

Throwing an exception is definitely a good idea, which I'd forgotten to 
do yet.  I will definitely make a point of doing so.

> Thanks, hope this is helpful.

Definitely, even though you are missing the context of the purpose of 
creating this method, well-intentioned feedback is always welcome.  I 
always appreciate a chance to explain something I have done, because one 
of  the best way to find flaws in your thinking is to explain it to others.

Insert policy design questionRick Gutleber27 Oct
Re: Insert policy design questionRick Gutleber27 Oct
  • Re: Insert policy design questionDrew M.27 Oct
    • Re: Insert policy design questionWarren Young27 Oct
      • Re: Insert policy design questionRick Gutleber28 Oct
        • Re: Insert policy design questionWarren Young28 Oct
          • Re: Insert policy design questionRick Gutleber28 Oct
            • Re: Insert policy design questionWarren Young28 Oct
              • Re: Insert policy design questionRick Gutleber28 Oct
                • Re: Insert policy design questionWarren Young28 Oct
                  • Re: Insert policy design questionRick Gutleber29 Oct
                    • SVN down?Rick Gutleber6 Nov
                      • Re: SVN down?Warren Young6 Nov
                  • Query::tellp( )Rick Gutleber29 Oct
                    • Re: Query::tellp( )Warren Young29 Oct
    • Re: Insert policy design questionRick Gutleber27 Oct
      • Re: Insert policy design questionDrew M.28 Oct
        • Re: Insert policy design questionRick Gutleber28 Oct
        • Re: Insert policy design questionWarren Young28 Oct
  • Re: Insert policy design questionWarren Young28 Oct
    • Re: Insert policy design questionRick Gutleber28 Oct