List:Falcon Storage Engine« Previous MessageNext Message »
From:Xuekun Hu Date:January 15 2009 3:30am
Subject:RE: Are multiple serial logs feasible or profitable?
View as plain text  
Hi, Guys

I implemented the Jim's idea on SRLUpdateRecords::append and SRLUpdateIndex::append. The
patch are attached. (Sorry for wrong coding style, this is only for concept proving. :-)
However the performance even drops ~5% on dbt2 with 10 warehouses and 32 connections.

In SRLUpdateRecords patch, I implemented a struct StreamList, singly linked list, to hold
the streams. First put all chilled records to the streamlist, then get the serial log
lock, copy ("logStream") to serial log, then out.
struct StreamList {
        Stream recordHeaderStream;      // hold the record's recordTableSpaceId,
record->recordNumber, priorVersion etc
        Stream recordStream;            // the chilled serial log record
        int32 sectionId, recordTableSpaceId;  // used to update record info during writing
to serial log
        RecordVersion *record;
        struct StreamList *next;
};
However it drops ~5% dbt2 performance. :-(
In previous version, I also used malloc to alloc buffer, instead of Stream. This version
patch didn't get performance drop, however also no improvement.

In SRLUpdateIndex patch, one stream is ok, to put all the variable-length index body to
segment, then copy to serial log. This patch has no big performance impact.

Any suggestions/comment are appreciated?

Thx, Xuekun

-----Original Message-----
From: Jim Starkey [mailto:jstarkey@stripped]
Sent: Tuesday, December 09, 2008 6:37 AM
To: Kevin Lewis
Cc: FalconDev
Subject: Re: Are multiple serial logs feasible or profitable?

Kevin Lewis wrote:
> One of our community testers has noticed that changing
> innodb_log_files_in_group from 2 to 6 gives dbt2 ~8%
> gain with 10 warehouses and 32 connections.
> It gives a hint that there is an optimization opportunity
> on a single serial log for Falcon which has lots of
> syncWrite contention, from SerialLog::flush,
> SRLUpdateIndex::append, and SRLUpdateRecords::append.
>
> >Ann Harrison noted;
> >One important aspect of the Serial Log is that it is
> >*serial* - events are logged in the order they occur.
> >It may be possible to maintain that property while
> > writing to two different files in parallel, but it
> > will certainly complicate managing the log.
> >We need to find another way to reduce contention.

Here's a starter.

Rather than building serial log messages directly, build them in a
separate stream, then get the serial log lock, copy the crud in, and get
out.  It's less efficient, but should reduce the time holding an
exclusive lock on the serial log, increasing concurrency.

Prototyping with SRLUpdateRecords and SRLUpdateRecords should be easy
and give a good whether the idea is worth pursuing.

--
Jim Starkey
President, NimbusDB, Inc.
978 526-1376


--
Falcon Storage Engine Mailing List
For list archives: http://lists.mysql.com/falcon
To unsubscribe:    http://lists.mysql.com/falcon?unsub=1


Thread
Are multiple serial logs feasible or profitable?Kevin Lewis8 Dec
  • Re: Are multiple serial logs feasible or profitable?Jim Starkey8 Dec
    • RE: Are multiple serial logs feasible or profitable?Xuekun Hu15 Jan
      • Re: Are multiple serial logs feasible or profitable?Jim Starkey15 Jan
        • RE: Are multiple serial logs feasible or profitable?Xuekun Hu16 Jan
        • RE: Are multiple serial logs feasible or profitable?Xuekun Hu20 Jan
          • Re: Are multiple serial logs feasible or profitable?Jim Starkey20 Jan
            • RE: Are multiple serial logs feasible or profitable?Xuekun Hu16 Feb