List:Falcon Storage Engine« Previous MessageNext Message »
From:Jim Starkey Date:December 15 2008 5:02pm
Subject:Re: Falcon and ZFS
View as plain text  
Kevin Lewis wrote:
> Jim Starkey wrote:
>> If Falcon were to allocate blocks of pages (say, 64K's worth) per 
>> data segmentmd per index and maybe section tree, we would get greater 
>> locality on very large page size file systems like ZFS, and probably 
>> not hurt us at all other systems.  It wouldn't have any affect on 99% 
>> of Falcon, and would probably be easy to prototype.
>
> Jim,
>
> I think this idea has been captured previously by
>
> WL#4564 - Falcon; Page-Allocation by groups.
>
> Let me know if you think this high level description needs to be 
> expanded or changed;
>
>  High-Level Description
>
> It is generally accepted that that reading 64K is effectively just as 
> fast as reading a 4K page.  Falcon contains an optional 'Sector cache' 
> which tries to take advantage of that fact.  This was added to version 
> 6.0.5 in June, 2008.  But in subsequent tests, we hardly ever found a 
> performance benefit.  In practical application, the overall cost of an 
> extra cache in front of the page cache was worse than the benefit of 
> having already read a few more pages than needed, allowing the engine 
> to skip another read every now and then.
>
> The page allocation scheme within a tablespace needs to be improved in 
> order to benefit from this sector cache.  We need to increase the 
> likelihood that consecutive pages on disk contain common data.   The 
> current scheme allocates one page at a time from the tablespace to a 
> particular table.  In a busy system, pages assigned to one table are 
> interleaved with pages assigned to completely different tables.  
> Falcon should assign some configurable number of pages to a table at a 
> time. This will greatly increase the chance that new pages with common 
> or dependent data are located closer together on disk.  And it would
> improve the possible payback of using the current 'sector cache' 
> scheme of read-ahead.
>
Yup, same idea.

-- 
Jim Starkey
President, NimbusDB, Inc.
978 526-1376

Thread
Falcon and ZFSJim Starkey15 Dec
  • Re: Falcon and ZFSKevin Lewis15 Dec
    • Re: Falcon and ZFSJim Starkey15 Dec