List:Backup« Previous MessageNext Message »
From:Ingo Strüwing Date:October 23 2009 11:07am
Subject:Re: WL#4056: Pluggable storage modules for backup
View as plain text  
Hi Rafal,

Rafal Somla, 23.10.2009 10:35:

> Ingo Strüwing wrote:
>> Rafal Somla, 22.10.2009 13:04:
>>> Ingo Strüwing wrote:
>>>> "Amount of data that has been written"
>> So you suggest asynchronous operation again as a use case.
> Yes, because as I wrote I think it is quite possible that storage module
> will drive some unusual physical storage device. Is this not a valid
> argument?

It is quite possible that a storage module will drive some unusual
physical storage device. That's a fact. But how is it a valid argument?
An argument for what? That we need asynchronous operation? In that case
I don't agree.

I may lack imagination. What could the possible options be for the
backup kernel to act on an "amount of data that has been written" less
than the requested amount?

1. Loop. Repeat the write with the remaining data. If the backup storage
module is allowed to return with less data than requested, and it
decided to do so for some reason before, it will probably return
immediately again with zero bytes written. We'd run in an infinite loop
until the "unusual physical storage device" returned to service. We
could add a sleep to avoid 100% CPU usage. But how long to sleep? Too
short and we consume too much resources on busy waiting. Too long and we
 lose time as the storage could be come available before we end sleeping.

2. Use the time and do something else in between? Our buffer is full. We
need to get rid of it before doing more data collection. Allocate more
buffers and fill them? If we really want to implement double buffering,
we would want to do that as a general solution with two or more threads.
What else could a backup kernel do with the available time?

3. (Add your ideas here)

My current standpoint is that there is no reasonable way to profit from
an "amount of data that has been written" information returned by the
storage module. Hence I request from the storage module that it does its
best to write the provided data unless it has to give up. If I want
asynchronism, I'd use threads. Or implement a full asynchronous
interface, which allows the kernel to wait until the storage signals
"ready for more data".

> Well, I'm still not convinced that my specification, as it is now, is
> ridiculous :)

I hope, you did not seriously read my comments as saying this. (I hope
the smiley transports this.) I oppose against the accentuation of one
possible status return in the specification. I mentioned other status'
that could as well be mentioned explicitly. I said that could could even
come up with more ridiculous examples.

The worst you could deduce from it is that I might find the mention of
"acknowledgement" in the high level specification ridiculous. But please
don't extend this to the whole specification.

But even for this piece of the HLS I wont use "ridiculous". What I think
about it is more like "ugly" or such. But as we are going towards
dropping the compression-selection service anyway, this isn't relevant
any more.

> I don't know what is your impression, but mine is that although this
> discussion is lengthy, we still clarify many things and understand
> better each ones position.

Agree. And it is perhaps good to spend much time in the specification.
At least better than noticing problems during coding only.

>   You also give some interesting suggestions I
> was not thinking about, like possibility to abort operation by storage
> module or interface for getting progress info from storage modules.

Yes, but this was to show that we don't try to cover all information
that could be transported over the interface. I do not suggest to
include these in the specification now. In contrast, I strive for
getting rid of the asynchronous option to make the interface even
simpler. I don't agree that this removes flexibility. I prefer to
specify asynchronism differently, and only later, if required at all.

Ingo Strüwing, Database Group
Sun Microsystems GmbH, Sonnenallee 1, D-85551 Kirchheim-Heimstetten
Geschäftsführer: Thomas Schröder,   Wolfgang Engels,   Wolf Frenkel
Vorsitzender des Aufsichtsrates: Martin Häring   HRB München 161028
Re: WL#4056: Pluggable storage modules for backupRafal Somla13 Oct
  • Re: WL#4056: Pluggable storage modules for backupIngo Strüwing15 Oct
    • Re: WL#4056: Pluggable storage modules for backupRafal Somla22 Oct
      • Re: WL#4056: Pluggable storage modules for backupIngo Strüwing22 Oct
        • RE: WL#4056: Pluggable storage modules for backupAndreas Almroth22 Oct
        • Re: WL#4056: Pluggable storage modules for backupRafal Somla23 Oct
          • Re: WL#4056: Pluggable storage modules for backupIngo Strüwing23 Oct
            • Re: WL#4056: Pluggable storage modules for backupRafal Somla3 Nov