List:General Discussion« Previous MessageNext Message »
From:Robert DiFalco Date:April 28 2006 7:35pm
Subject:Streaming LOB Data
View as plain text  
It appears (for InnoDB at least) that while INSERTing a LOB that all LOB
data must be loaded into memory before it is written to disk. Or is it
just the size of the combined log files? Looking at Task Manager, it
looks like it DOES try to load the whole thing into memory before
streaming it to disk; which just seems wacky.

The error I get is this:

	"Out of memory (Needed xxx ..."

If the server crashes I get this:

InnoDB: which exceeds the log group capacity 18870682.
InnoDB: If you are using big BLOB or TEXT rows, you must set the
InnoDB: combined size of log files at least 10 times bigger than the
InnoDB: largest such row.
060427 15:26:53  InnoDB: Error: cannot allocate 539001144 bytes of
InnoDB: memory with malloc! Total allocated memory
InnoDB: by InnoDB 52823786 bytes. Operating system errno: 8
InnoDB: Check if you should increase the swap file or
InnoDB: ulimits of your operating system.
InnoDB: On FreeBSD check you have compiled the OS with
InnoDB: a big enough maximum process size.
InnoDB: Note that in most 32-bit computers the process
InnoDB: memory space is limited to 2 GB or 4 GB.

How can I make the size of LOBs I insert NOT be memory constrained? 


Streaming LOB DataRobert DiFalco28 Apr