List:Commits« Previous MessageNext Message »
From:Daogang Qu Date:January 25 2011 8:58am
Subject:Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925
View as plain text  
2011-01-25 16:29, anders wrote:
> On Tue, 2011-01-25 at 15:55 +0800, Daogang Qu wrote:
>    
>> 2011-01-25 14:51, anders wrote:
>>      
>>> Hi Daogang,
>>>
>>> [snip]
>>>
>>>        
>>>>>>
>>>>>>              
>>>>>>> 3. Isn't there any race condition between rotation and
> pre-allocation?
>>>>>>>
>>>>>>>        server --->     decides to use master.X
>>>>>>>
>>>>>>>        mysqlbinlogalloc --->     creates master.X
>>>>>>>
>>>>>>>        server --->     fails while creating master.X
> (booom)
>>>>>>>
>>>>>>>
>>>>>>>                
>>>>> Not sure if it will fail, but this case exists, I think.
>>>>> A way to avoid this case is that mysqlbinlogalloc create the file
> with a
>>>>> temporary name and then rename it.
>>>>>
>>>>>
>>>>>            
>>>> This case is not exist. Because we assumed that user just can
>>>> do statical preallocation  when server is off. The automatic
>>>> preallocation will be used  when server is on.  I think we should
>>>> design to forbid user from doing statically preallocation when
>>>> server is on. Maybe just documentation is not enough. What
>>>> do you think?
>>>>
>>>>          
>>> It is not convenient if user can preallocation the binlog file only when
>>> server is off. I don't think users can plan all things well before
>>> starting the server. So let us make the facility more convenient.
>>>
>>>        
>> DBA just can statically preallocate binlog files when the servers are
>> off and DBA
>> can automatically preallocate binlog files when the servers are on. We
>> don't need
>> DBA do all things well before server start. We just need DBA do the
>> static preallocation
>> if DBA want. We should use automatic preallocation as most as possible
>> instead of
>> static preallocation. So I don't think it will bring inconvenience.
>>      
> Think over the cases,
> DBA don't know the new option, so the option is not set when starting
> the server. However, the DBA learned the option and mysqlbinlogalloc
> from other DBAs. And then he has to restart the server if he want to
> use these functions. But it is not easy to restart a db server in many
> information systems.
>    
I don't think it's strong enough to do static preallocation when server 
is on.
> Another case:
> posix_fallocate is not implemented on some OSs. The DBA doesn't want
> server to preallocate binary log file, as this  will effect the mysql
> server's performance. But he still want to preallocate some binary files
> manually.
>    
DBA can statically preallocate binlog files when server is off. The 
static preallocation still
will effect the server's performance if we statically preallocate binlog 
files when server is on.

Best Regards,

Daogang
>>>>>>>
>>>>>>>                
>>>>>> No. Static and automatic preallocation are compatible. One will
> not
>>>>>> preallocate the file if another is a preallocating it. So no
> conflict here.
>>>>>>
>>>>>>
>>>>>>              
>>>>> [snip]
>>>>>
>>>>>
>>>>>            
>>>>>>
>>>>>>              
>>>>>>> 4. Using the binlog_preallocate to decide if either the old
> or new behavior should
>>>>>>>        be used is not a good idea. We may pre-allocate files
> and start the server with
>>>>>>>          binlog_preallocate<>     0 and then forget and
> restart the server with binlog_preallocate
>>>>>>>        = 0. In this case, we are going to have dangling files
> consuming lots of disk space.
>>>>>>>
>>>>>>>        If We can I think we must avoid situations where an
> user's mistake may cause problems.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                
>>>>>> Maybe. But how to avoid this?
>>>>>>
>>>>>>
>>>>>>              
>>>>>
>>>>>            
>>>>>>> 5. Using binlog_preallocate>     1 to automatically
> pre-allocate binary log files is not
>>>>>>>        a good idea because users will not have control when
> the pre-allocation will happen.
>>>>>>>        See item 1.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                
>>>>>> I don't think the user should control something in the process as
> it's a
>>>>>> automatic preallocation.
>>>>>>
>>>>>>
>>>>>>              
>>>>>>> 6. Why do you need to keep the old behavior where new files
> are created after searching
>>>>>>>        in the filesystem? I know that this done just for
> safety in order to avoid overwriting
>>>>>>>        a possible useful binary log. However, this case will
> only happen if someone manually
>>>>>>>        copies files around.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                
>>>>>> The old behavior is good. It can guarantee that Static and
> automatic
>>>>>> preallocation are compatible.
>>>>>>
>>>>>>
>>>>>>              
>>>>>>>        Besides, we have committed several patches towards
> making the index file the source
>>>>>>>        of truth. So why don't we keep just the new behavior
> where the index file is the source
>>>>>>>        of truth?
>>>>>>>
>>>>>>>        If overwriting a file is really a concern. I think we
> should use two states in the
>>>>>>>        binary log:
>>>>>>>          . LOG_EVENT_BINLOG_IN_USE_F
>>>>>>>          . LOG_EVENT_BINLOG_PRE_ALLOC
>>>>>>>
>>>>>>>        So, if the new file is pre-allocated, its status would
> be LOG_EVENT_BINLOG_PRE_ALLOC
>>>>>>>        and the server would be able to know if it is safe to
> overwrite it or not.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                
>>>>> I agree with Alfranio. binlog_preallocate is not necessary.
>>>>> There are other ways that can keep the both behaviors.
>>>>> 1. Just like Alfranio said.
>>>>> 2. Use a new BINLOG_MAGIC
>>>>> 3. Add a prefix in the file name.
>>>>>       /mysqlbinlogalloc  --start=1 --end=10 --alloc-size=2G
> --binlog-name=mysql-binlog
>>>>>
>>>>> pre-mysql-binlog.000001 .. pre-mysql-binlog.000010 are generated.
>>>>>
>>>>>            
>>> Add a prefix is convenient to users too. Users is easy to know and
>>> manage these files.
>>>
>>>        
>> The point is that we don't need users to manage these binlog files. All
>> the things
>> will be handled automatically.
>>      
>>>>> When server need a new binlog file, it will try to find a
>>>>> pre-mysql-binlog.* and rename it to a regular binlog name.
>>>>>
>>>>> I prefer 3.
>>>>>
>>>>>
>>>>>            
>>>> The number 3 just can handle the case of static preallocation if
>>>> binlog_preallocate is not
>>>> used. But it can't support automatic preallocation. In fact static
>>>> preallocation just is a assistant
>>>> tool.
>>>>
>>>> I persist to use binlog_preallocate for providing more choice for users
>>>> and meet different
>>>> requirement from users.
>>>>
>>>>
>>>>          
>>> "- if binlog_preallocate>= 1, creating and preallocating more binlog
>>> files. The
>>> number of preallocated binlogs are equivalent to binlog_preallocate
>>> value."
>>> I think it is not necessary to pre-allocate more than one file when
>>> server needs a new binary log file.
>>> So binlog_preallocate = 0 means server doesn't pre-allocate space for
>>> binary log file.
>>> binlog_preallocate = 1 means server pre-allocates space for binary log
>>> file.
>>>
>>> Server should use pre-allocated binary log file if there are some,
>>> whenever binlog_preallocate is 0 or 1. In this way, mysqlbinlogalloc can
>>> be used when servers are on.
>>>
>>>        
>> In fact, DBA should not overuse mysqlbinlogalloc, which just is a
>> assistant tool. DBA
>> should use it as least as possible. I don't think we should avoid using
>> simple options
>> just because we are afraid of the misuse of DBA. Let's have a discussion
>> with Mats
>> and Alfranio today. Thanks!
>>
>> Best Regards,
>>
>> Daogang
>>      
>>>
>>>        
>>
>>      
>    

Thread
bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925daogang.qu5 Jan
  • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925Alfranio Correia21 Jan
    • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925Daogang Qu24 Jan
      • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925Luís Soares24 Jan
      • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925anders24 Jan
        • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925Alfranio Correia24 Jan
          • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925Daogang Qu25 Jan
        • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925Daogang Qu25 Jan
          • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925anders25 Jan
            • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925Daogang Qu25 Jan
              • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925anders25 Jan
                • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925Daogang Qu25 Jan
          • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925Alfranio Correia25 Jan
      • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925Alfranio Correia24 Jan
        • Re: bzr commit into mysql-trunk branch (daogang.qu:3446) WL#4925Daogang Qu25 Jan