List:Internals« Previous MessageNext Message »
From:Mats Kindahl Date:May 12 2009 7:10pm
Subject:Re: MySQL Reengineering Project
View as plain text  

Alaric Snell-Pym wrote:
> 
> On 12 May 2009, at 4:16 pm, Mats Kindahl wrote:
> 
>> However, do you know of any other interfaces that work this way? I
>> am personally
>> not aware of any other, but then I don't know every corner of the
>> code like Serg
>> does. :)
> 
> Yes; when tracing up into the caller within mysql to find out why our
> rnd_next's results were being ignored in joins, we found (IIRC in a
> file called join.cc) the code that handles joins, which revolved
> around JOIN * and JOIN_TAB * arguments being passed around, which were
> also mutated; the JOIN and JOIN_TAB seemed to be initialised with the
> proposed structure of the join (presumably what EXPLAIN returns), but
> then also store the in-progress state of the join (storing candidate
> rows).

Yup. This is a major pain-point when it comes to parallelizing queries as well.
In order to be able to do that it is necessary to disassociate state information
from the "execution plan," and this is not the only place.

> That, combined with the fact that in my recent foray into UDFs I found
> that string UDFs are passed a buffer with room for 255 characters to
> put their result into, and then have the choice of using it and
> returning a pointer to it or mallocing their own buffer and returning
> a pointer to that (meaning that the caller, somewhere, must have code
> to the effect of "if (returned_ptr != allocated_buffer)
> free(returned_ptr)" to clear up, I guess?), led me to suspect that
> there was an ingrained culture of functions plopping their results
> into spaces left for them somewhere. Ok, there's often good reasons
> for string-processing functions to put their results into preallocated
> buffers, but I suspect in this case somebody originally wrote it that
> way because it was the ingrained culture, and then later had to add
> the ability to malloc your own buffer (wasting the pre-allocated
> buffer) in order to handle strings of more than 255 characters :-)

I didn't know about that. Thanks!

/me marks it down on his Big List of Things

> Although I may have found the only examples of this sort of thing in a
> very large and otherwise much more cleanly-structured codebase for all
> I know!

I'm sure this is not the only issue. :)

And for cleanly structured... well... I think it lacks quite a lot. :)

Best wishes,
Mats Kindahl
-- 
Mats Kindahl
Senior Software Engineer
Database Technology Group
Sun Microsystems
Thread
MySQL Reengineering ProjectManyi Lu - Sun Norway12 May
  • Re: MySQL Reengineering ProjectJay Pipes12 May
  • Re: MySQL Reengineering ProjectMARK CALLAGHAN12 May
    • Re: MySQL Reengineering ProjectMats Kindahl12 May
      • Re: MySQL Reengineering ProjectManyi Lu - Sun Norway12 May
        • Re: MySQL Reengineering ProjectMats Kindahl12 May
    • Re: MySQL Reengineering ProjectStewart Smith13 May
  • Re: MySQL Reengineering ProjectMikiya Okuno18 May
    • Re: MySQL Reengineering ProjectSergei Golubchik18 May
      • Re: MySQL Reengineering ProjectRoy Lyseng18 May
Re: MySQL Reengineering ProjectAlaric Snell-Pym12 May
  • Re: MySQL Reengineering ProjectMats Kindahl12 May
    • Re: MySQL Reengineering ProjectJay Pipes12 May
      • Re: MySQL Reengineering ProjectJay Pipes12 May
        • Re: MySQL Reengineering ProjectAlex Esterkin12 May
          • Re: MySQL Reengineering ProjectBrian Aker12 May
      • Re: MySQL Reengineering ProjectSergei Golubchik12 May
        • Re: MySQL Reengineering ProjectBrian Aker12 May
          • Re: MySQL Reengineering ProjectSergei Golubchik12 May
            • Re: MySQL Reengineering ProjectAlex Esterkin12 May
              • Re: MySQL Reengineering ProjectBrian Aker12 May
            • Re: MySQL Reengineering ProjectJay Pipes12 May
              • Re: MySQL Reengineering ProjectDavi Arnaut13 May
              • Re: MySQL Reengineering ProjectAlex Esterkin13 May
                • Re: MySQL Reengineering ProjectJay Pipes13 May
            • Re: MySQL Reengineering ProjectMichael Widenius14 May
          • Re: MySQL Reengineering ProjectJonas Oreland13 May
        • Re: MySQL Reengineering ProjectRoy Lyseng12 May
      • Re: MySQL Reengineering ProjectMats Kindahl12 May
    • Re: MySQL Reengineering ProjectAlaric Snell-Pym12 May
      • Re: MySQL Reengineering ProjectMats Kindahl12 May
      • Re: MySQL Reengineering ProjectMichael Widenius14 May
        • Re: MySQL Reengineering ProjectAlaric Snell-Pym15 May
          • Re: MySQL Reengineering ProjectKristian Nielsen15 May
            • Re: MySQL Reengineering ProjectBrian Aker15 May
            • Re: MySQL Reengineering ProjectAlaric Snell-Pym15 May
          • Re: MySQL Reengineering ProjectMichael Widenius6 Jun
Re: MySQL Reengineering ProjectMats Kindahl12 May
Re: MySQL Reengineering ProjectAlex Esterkin12 May
  • Re: MySQL Reengineering ProjectMasood Mortazavi13 May
    • Re: MySQL Reengineering ProjectBrian Aker13 May
    • Re: MySQL Reengineering ProjectAlex Esterkin13 May
      • Re: MySQL Reengineering ProjectBaron Schwartz13 May
        • Re: MySQL Reengineering ProjectStewart Smith13 May
          • Re: MySQL Reengineering ProjectBaron Schwartz13 May
    • Re: MySQL Reengineering ProjectAlex Esterkin13 May
    • Re: MySQL Reengineering ProjectStewart Smith13 May
  • Re: MySQL Reengineering ProjectMichael Widenius14 May