List:MySQL++« Previous MessageNext Message »
From:Drew M. Date:October 28 2008 7:44am
Subject:Re: Insert policy design question
View as plain text  
Whew, lots of text flying back and forth. I will try to condense my
responses:
Rick wrote:
>I believe you are misinterpreting the code.
Yes sorry I did misread the code. I was too busy getting my hackles up about
other items I mentioned previously. ;)

Rick wrote:
>The problem with that is that there is then really nothing for the
InsertPolicy to do.  If it doesn't maintain the actual query it cannot
determine whether another object can be added.
Exactly. If the policy just dictates when it is ok to add more elements and
when it is not, you don't have to replicate the query-building logic in each
Policy class. You can code up an arbitrary policy and only have to worry
about the actual policy logic. The approach as coded, on the other hand,
requires that if I want to create a different policy (or even just the three
originally envisioned), I have to add the query-building logic separately
each time, and then have to manage this if I make any changes to the Query
class later.

Warren wrote:
>If any one of them fails, do you really want to leave half of the
container's data still in the DB?  Query::insert() either succeeds or fails;
it never inserts half a record.  I think Query::insertfrom() should behave
the same way, at least by default.

and Rick wrote:
>  If you call insertfrom( ), unlike the other methods in mysqlpp::Query it
is possible for the method to fail _partway_ through, i.e., some records are
inserted, but not all.   It only makes sense that you use a transaction,
because otherwise the result of a failed operation is indeterminate, which
is always Very Bad.

My problem isn't so much with employing transactions, rather it's that mysql
transactions can't, to my knowledge, be nested. So, if you have a complex
transaction and then call Query::insertfrom(), you have committed your
previous statements regardless of whether Query::insertfrom() fails or not,
and can't rollback. This is what I meant by an unavoidable situation for
developers using mysql++. A bool passed to the function would allow
developers to avoid this situation, but it just seems awkward to me.

Warren wrote:
>We may not want to bother creating MaxPacketPolicy for this first version.
I don't really have a horse in that race, but imo you should be able to
determine any sort of policy required (max query string size, max elements)
if you pass a) the Query object (or rather, the query string), and b) the
element to insert (or rather, the string to insert into the query). My
assumption has been that each Policy object would have internal state
variables to assist.

Cheers.

On Mon, Oct 27, 2008 at 3:58 PM, Rick Gutleber <rgutleber@stripped> wrote:

> Drew M. wrote:
>
>> The other code in query.h uses an overloaded << operator with a Query
>> object
>> on the left side. Your code uses an ostringstream, which is why the
>> behavior
>> is different.
>>
>>
> If you look at the code in row.h, from which the SSQLS-generated classes
> are derived, for the equal_list and value_list templates, you will see that
> they are designed to be used with C++ streams in general.  There is nothing
> specific to Query as far as I understand.  Certainly there is no overloaded
> << operator in query.h.  I thought perhaps there was something specific to
> Query at first, but was unable to find any reason in the code why there
> should be.
>
> Of course, this is all predicated on the fact that I may have missed or
> misunderstood something, as I am still learning MySQL++ and it's not an
> unsophisticated piece of code.
>
>  Some constructive feedback for discussion/consideration: I suggest you
>> make
>> your code conform more to the same style as the other insert-ish methods
>> in
>> Query.
>>
> You might wish to peruse the mailing list history which contains our entire
> discussion from earlier in October.  Your feedback is certainly appreciated,
> and perhaps you might have something else to add to or improve in the
> original design.
> The InsertPolicy concept was Warren's idea because he didn't want to limit
> the criteria used to determine how bulk INSERT statements are built.
> The constraints are the key difference between the insertfrom( ) method and
> the regular insert( iter1, iter2 ) method.  The topic that caused the
> initial discussion for this whole thing is the MySQL "max packet size"
> setting which determines how big a single SQL statement sent to MySQL can
> be.  I had heretofore been "manually" constraining the size of the INSERT
> statement at the application level, which is something I think could be
> transparently handled at the library level, since a major selling point of
> MySQL++ is that it frees you from the tedium of building workaday SQL
> statements.  Warren agreed that this could be a useful addition and
> suggested a way I could implement it.  Frankly, it took me quite a while to
> understand exactly what he meant (assuming I actually have done so ;-)), and
> I believe the prototypical implementation I have provided (it's not intended
> to be complete), accurately expresses Warren's idea in a way I think is
> useful.
>
>
>  To do this, you would actually remove the add() method and keep the
>> query-writing logic in the query class. Rather than have your policy
>> object
>> construct the query, you would construct the query within the insertfrom()
>> call and then iterate through the list.
>>
>
> The problem with that is that there is then really nothing for the
> InsertPolicy to do.  If it doesn't maintain the actual query it cannot
> determine whether another object can be added.  Passing the query in to each
> canAdd( ) call is another possibility, but given that you can construct a
> quoted query using any C++ stream (according to my understanding) you might
> as well let the InsertPolicy handle it.  Another possibility that just
> occurred to me is creating the query in the InsertPolicy allows for
> supporting autoincrement ID columns, which SSQLS does not otherwise do.
>
>
>  I am also curious if you meant to
>> add an entire insert statement each time the policy's add() method is
>> called.
>>
>
> I believe you are misinterpreting the code.  Each call to add( ) adds an
> additional value list to the INSERT statement being maintained by the
> InsertPolicy object.  It only creates a new INSERT statement immediately
> subsequent to a call to startQuery( ).
>
>  It seems like you would keep adding to the same statement until the
>> policy decided that you had iterated enough (or the end of the list).
>>
>>
>
> The idea behind the insert policy object was that there are special
> constraints that might apply to creating a bulk INSERT statement.  One
> obvious
> constraint is the MySQL "max packet size" parameter I described earlier.
>  Warren's idea for the InsertPolicy was that there could be other
> constraints that users might wish to impose themselves using hand-rolled
> InsertPolicies.  MySQL++ would provide the 3 or 4 "obvious" policy objects,
> but there would be nothing stopping a user from passing any object
> conforming to the InsertPolicy interface expressed in the insertfrom( )
> template method.
>
>
>  I think you should take the transaction calls out, similar to the other
>> insert-ish methods.
>>
>
> Again, this was Warren's suggestion, and I think it's not only a good idea,
> but absolutely necessary.  If you call insertfrom( ), unlike the other
> methods in mysqlpp::Query it is possible for the method to fail _partway_
> through, i.e., some records are inserted, but not all.   It only makes sense
> that you use a transaction, because otherwise the result of a failed
> operation is indeterminate, which is always Very Bad.
>
>
>  It should be up to the end-developer to decide whether
>> or not to employ transactions, and using them in library code might cause
>> unintended (and unavoidable) effects for end-developers.
>>
>>
>
> Normally, I would agree, but for the reasons above, I don't think it's
> avoidable.  This method is, as with most parts of MySQL++, only one way to
> do a particular operation.  Given proper caveats in the documentation, the
> user can determine whether this method would cause a problem, and if so,
> there are plenty of other, albeit less convenient, ways to achieve the same
> end.
>
>
>  I would suggest instead throwing an exception (if the query has exceptions
>> enabled, that is). This is more in line with the rest of the Query class.
>>
>>
>
> Throwing an exception is definitely a good idea, which I'd forgotten to do
> yet.  I will definitely make a point of doing so.
>
>
>  Thanks, hope this is helpful.
>>
>>
>
> Definitely, even though you are missing the context of the purpose of
> creating this method, well-intentioned feedback is always welcome.  I always
> appreciate a chance to explain something I have done, because one of  the
> best way to find flaws in your thinking is to explain it to others.
>
>
>

Thread
Insert policy design questionRick Gutleber27 Oct
Re: Insert policy design questionRick Gutleber27 Oct
  • Re: Insert policy design questionDrew M.27 Oct
    • Re: Insert policy design questionWarren Young27 Oct
      • Re: Insert policy design questionRick Gutleber28 Oct
        • Re: Insert policy design questionWarren Young28 Oct
          • Re: Insert policy design questionRick Gutleber28 Oct
            • Re: Insert policy design questionWarren Young28 Oct
              • Re: Insert policy design questionRick Gutleber28 Oct
                • Re: Insert policy design questionWarren Young28 Oct
                  • Re: Insert policy design questionRick Gutleber29 Oct
                    • SVN down?Rick Gutleber6 Nov
                      • Re: SVN down?Warren Young6 Nov
                  • Query::tellp( )Rick Gutleber29 Oct
                    • Re: Query::tellp( )Warren Young29 Oct
    • Re: Insert policy design questionRick Gutleber27 Oct
      • Re: Insert policy design questionDrew M.28 Oct
        • Re: Insert policy design questionRick Gutleber28 Oct
        • Re: Insert policy design questionWarren Young28 Oct
  • Re: Insert policy design questionWarren Young28 Oct
    • Re: Insert policy design questionRick Gutleber28 Oct