Open Stack

I don't think cinder is lying. Firstly, the "backup create" dialog states:

> Backups will be the same size as the volume they originate from.

Secondly, I believe it highly depends on the actual storage backend  
and its implementation how to create a backup. On a different storage  
backend it might look completely different. We're on ceph from the  
beginning so I can't comment on alternatives.

As for your question, how exactly did your dd command look like? Did  
you fill up the file with zeroes (dd if=/dev/zero)? In that case the  
low used bytes number would make sense, if you filled it up randomly  
the du size should be higher.

Zitat von Satish Patel <satish.txt at gmail.com>:

> Thank you Eugen,
>
> I am noticing similar thing what you noticed. That means cinder lying to
> use or doesn't know how to calculate size of copy-on-write.
>
> One more question, I have created 1G file using dd command and took
> incremental backup and found ceph only showing 28 MiB size of backup. Does
> that sound right?
>
>
> root at ceph1:~# rbd -p backups du NAME PROVISIONED USED
> volume-285a49a6-0e03-49e5-abf1-1c1efbfeb5f2.backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc at backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc.snap.1684260707.1682937
> 10 GiB 68 MiB
> volume-285a49a6-0e03-49e5-abf1-1c1efbfeb5f2.backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc at backup.294b58af-771b-4a9f-bb7b-c37a4f84d678.snap.1684260787.943873
> 10 GiB 36 MiB
> volume-285a49a6-0e03-49e5-abf1-1c1efbfeb5f2.backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc at backup.c9652662-36bd-4e74-b822-f7ae10eb7246.snap.1684330702.6955926
> 10 GiB 28 MiB
> volume-285a49a6-0e03-49e5-abf1-1c1efbfeb5f2.backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc
> 10 GiB 0 B
>
> On Wed, May 17, 2023 at 9:26 AM Eugen Block <eblock at nde.ag> wrote:
>
>> Hi,
>>
>> just to visualize Rajat's response, ceph creates copy-on-write
>> snapshots so the incremental backup doesn't really use much space. On
>> a Victoria cloud I created one full backup of an almost empty volume
>> (made an ext4 filesystem and mounted it, create one tiny file), then
>> created a second tiny file and then made another incremental backup,
>> this is what ceph sees:
>>
>> ceph01:~ # rbd du <backup_pool>/volume-6662f50a-a74c-47a4-8abd-a49069f3614c
>> NAME
>>                                            PROVISIONED  USED
>>
>> volume-6662f50a-a74c-47a4-8abd-a49069f3614c at backup.650a4f8f-7b61-447e-9eb9-767c74b15342.snap.1684329174.8419683
>>       5 GiB  192
>> MiB
>>
>> volume-6662f50a-a74c-47a4-8abd-a49069f3614c at backup.1d358548-5d1d-4e03-9728-bb863c717910.snap.1684329450.9599462
>>       5 GiB   16
>> MiB
>> volume-6662f50a-a74c-47a4-8abd-a49069f3614c
>>                                                  5 GiB      0 B
>> <TOTAL>
>>
>> backup.650a4f8f-7b61-447e-9eb9-767c74b15342 (using 192 MiB) is the
>> full backup, backup.1d358548-5d1d-4e03-9728-bb863c717910 is the
>> incremental backup (using 16 MiB).
>>
>> Zitat von Rajat Dhasmana <rdhasman at redhat.com>:
>>
>> > Hi Satish,
>> >
>> > Did you check the size of the actual backup file in ceph storage? It
>> should
>> > be created in the *backups* pool[1].
>> > Cinder shows the same size of incremental backup as a normal backup but
>> > file size should be different from
>> > the size shown in cinder DB records. Also file size of incremental backup
>> > should not be the same as the file size of full backup.
>> >
>> > [1]
>> >
>> https://github.com/openstack/devstack/blob/34afa91fc9f830fc8e1fdc4d76e7aa6d4248eaaa/lib/cinder_backups/ceph#L22
>> >
>> > Thanks
>> > Rajat Dhasmana
>> >
>> > On Wed, May 17, 2023 at 12:25 AM Satish Patel <satish.txt at gmail.com>
>> wrote:
>> >
>> >> Folks,
>> >>
>> >> I have ceph storage for my openstack and configure cinder-volume and
>> >> cinder-backup service for my disaster solution. I am trying to use the
>> >> cinder-backup incremental option to save storage space but somehow It
>> >> doesn't work the way it should work.
>> >>
>> >> Whenever I take incremental backup it shows a similar size of original
>> >> volume. Technically It should be smaller. Question is does ceph
>> >> support incremental backup with cinder?
>> >>
>> >> I am running a Yoga release.
>> >>
>> >> $ openstack volume list
>> >>
>> +--------------------------------------+------------+------------+------+-------------------------------------+
>> >> | ID                                   | Name       | Status     |
>> >> Size | Attached to                         |
>> >>
>> +--------------------------------------+------------+------------+------+-------------------------------------+
>> >> | 285a49a6-0e03-49e5-abf1-1c1efbfeb5f2 | spatel-vol | backing-up |
>> >>  10 | Attached to spatel-foo on /dev/sdc  |
>> >>
>> +--------------------------------------+------------+------------+------+-------------------------------------+
>> >>
>> >> ### Create full backup
>> >> $ openstack volume backup create --name spatel-vol-backup spatel-vol
>> --force
>> >> +-------+--------------------------------------+
>> >> | Field | Value                                |
>> >> +-------+--------------------------------------+
>> >> | id    | 4351d9d3-85fa-4cd5-b21d-619b3385aefc |
>> >> | name  | spatel-vol-backup                    |
>> >> +-------+--------------------------------------+
>> >>
>> >> ### Create incremental
>> >> $ openstack volume backup create --name spatel-vol-backup-1
>> >> --incremental --force spatel-vol
>> >> +-------+--------------------------------------+
>> >> | Field | Value                                |
>> >> +-------+--------------------------------------+
>> >> | id    | 294b58af-771b-4a9f-bb7b-c37a4f84d678 |
>> >> | name  | spatel-vol-backup-1                  |
>> >> +-------+--------------------------------------+
>> >>
>> >> $ openstack volume backup list
>> >>
>> +--------------------------------------+---------------------+-------------+-----------+------+
>> >> | ID                                   | Name                |
>> >> Description | Status    | Size |
>> >>
>> +--------------------------------------+---------------------+-------------+-----------+------+
>> >> | 294b58af-771b-4a9f-bb7b-c37a4f84d678 | spatel-vol-backup-1 | None
>> >>        | available |   10 |
>> >> | 4351d9d3-85fa-4cd5-b21d-619b3385aefc | spatel-vol-backup   | None
>> >>        | available |   10 |
>> >>
>> +--------------------------------------+---------------------+-------------+-----------+------+
>> >>
>> >>
>> >> My incremental backup still shows 10G size which should be lower
>> >> compared to the first backup.
>> >>
>> >>
>> >>
>>
>>
>>
>>
>>