<div dir="ltr">You are goddamn right :) <div><br></div><div>I was using /dev/zero before. After using /dev/random I can see the correct representation 1.1 GiB size of the last incremental backup. </div><div><br></div><span style="color:rgb(0,0,0);font-family:"Bitstream Vera Sans Mono",monospace;font-size:13px;white-space:pre;background-color:rgb(240,240,240)">root@ceph1:~# rbd -p backups du
NAME PROVISIONED USED
volume-285a49a6-0e03-49e5-abf1-1c1efbfeb5f2.backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc@backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc.snap.1684260707.1682937 10 GiB 68 MiB
volume-285a49a6-0e03-49e5-abf1-1c1efbfeb5f2.backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc@backup.294b58af-771b-4a9f-bb7b-c37a4f84d678.snap.1684260787.943873 10 GiB 36 MiB
volume-285a49a6-0e03-49e5-abf1-1c1efbfeb5f2.backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc@backup.c9652662-36bd-4e74-b822-f7ae10eb7246.snap.1684330702.6955926 10 GiB 28 MiB
volume-285a49a6-0e03-49e5-abf1-1c1efbfeb5f2.backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc@backup.7e9a48db-b513-40d8-8018-f73ef52cb025.snap.1684331929.0653753 10 GiB 1.1 GiB
</span><div><br></div><div>This is just confusion when I look at the backup list, They all say 10G. I wish it would bring actual numbers from ceph instead of 10G. I can understand but it's hard to explain that to customers :( </div><div><br></div><div><span style="color:rgb(0,0,0);font-family:"Bitstream Vera Sans Mono",monospace;font-size:13px;white-space:pre;background-color:rgb(240,240,240)"># openstack volume backup list
+--------------------------------------+---------------------+-------------+-----------+------+
| ID | Name | Description | Status | Size |
+--------------------------------------+---------------------+-------------+-----------+------+
| ec141929-cc74-459a-b8e7-03f016df9cec | spatel-vol-backup-4 | None | available | 10 |
| 7e9a48db-b513-40d8-8018-f73ef52cb025 | spatel-vol-backup-3 | None | available | 10 |
| c9652662-36bd-4e74-b822-f7ae10eb7246 | spatel-vol-backup-2 | None | available | 10 |
| 294b58af-771b-4a9f-bb7b-c37a4f84d678 | spatel-vol-backup-1 | None | available | 10 |
| 4351d9d3-85fa-4cd5-b21d-619b3385aefc | spatel-vol-backup | None | available | 10 |
+--------------------------------------+---------------------+-------------+-----------+------+
</span></div><div> </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, May 17, 2023 at 9:49 AM Eugen Block <<a href="mailto:eblock@nde.ag">eblock@nde.ag</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I don't think cinder is lying. Firstly, the "backup create" dialog states:<br>
<br>
> Backups will be the same size as the volume they originate from.<br>
<br>
Secondly, I believe it highly depends on the actual storage backend <br>
and its implementation how to create a backup. On a different storage <br>
backend it might look completely different. We're on ceph from the <br>
beginning so I can't comment on alternatives.<br>
<br>
As for your question, how exactly did your dd command look like? Did <br>
you fill up the file with zeroes (dd if=/dev/zero)? In that case the <br>
low used bytes number would make sense, if you filled it up randomly <br>
the du size should be higher.<br>
<br>
Zitat von Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>>:<br>
<br>
> Thank you Eugen,<br>
><br>
> I am noticing similar thing what you noticed. That means cinder lying to<br>
> use or doesn't know how to calculate size of copy-on-write.<br>
><br>
> One more question, I have created 1G file using dd command and took<br>
> incremental backup and found ceph only showing 28 MiB size of backup. Does<br>
> that sound right?<br>
><br>
><br>
> root@ceph1:~# rbd -p backups du NAME PROVISIONED USED<br>
> volume-285a49a6-0e03-49e5-abf1-1c1efbfeb5f2.backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc@backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc.snap.1684260707.1682937<br>
> 10 GiB 68 MiB<br>
> volume-285a49a6-0e03-49e5-abf1-1c1efbfeb5f2.backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc@backup.294b58af-771b-4a9f-bb7b-c37a4f84d678.snap.1684260787.943873<br>
> 10 GiB 36 MiB<br>
> volume-285a49a6-0e03-49e5-abf1-1c1efbfeb5f2.backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc@backup.c9652662-36bd-4e74-b822-f7ae10eb7246.snap.1684330702.6955926<br>
> 10 GiB 28 MiB<br>
> volume-285a49a6-0e03-49e5-abf1-1c1efbfeb5f2.backup.4351d9d3-85fa-4cd5-b21d-619b3385aefc<br>
> 10 GiB 0 B<br>
><br>
> On Wed, May 17, 2023 at 9:26 AM Eugen Block <<a href="mailto:eblock@nde.ag" target="_blank">eblock@nde.ag</a>> wrote:<br>
><br>
>> Hi,<br>
>><br>
>> just to visualize Rajat's response, ceph creates copy-on-write<br>
>> snapshots so the incremental backup doesn't really use much space. On<br>
>> a Victoria cloud I created one full backup of an almost empty volume<br>
>> (made an ext4 filesystem and mounted it, create one tiny file), then<br>
>> created a second tiny file and then made another incremental backup,<br>
>> this is what ceph sees:<br>
>><br>
>> ceph01:~ # rbd du <backup_pool>/volume-6662f50a-a74c-47a4-8abd-a49069f3614c<br>
>> NAME<br>
>> PROVISIONED USED<br>
>><br>
>> volume-6662f50a-a74c-47a4-8abd-a49069f3614c@backup.650a4f8f-7b61-447e-9eb9-767c74b15342.snap.1684329174.8419683<br>
>> 5 GiB 192<br>
>> MiB<br>
>><br>
>> volume-6662f50a-a74c-47a4-8abd-a49069f3614c@backup.1d358548-5d1d-4e03-9728-bb863c717910.snap.1684329450.9599462<br>
>> 5 GiB 16<br>
>> MiB<br>
>> volume-6662f50a-a74c-47a4-8abd-a49069f3614c<br>
>> 5 GiB 0 B<br>
>> <TOTAL><br>
>><br>
>> backup.650a4f8f-7b61-447e-9eb9-767c74b15342 (using 192 MiB) is the<br>
>> full backup, backup.1d358548-5d1d-4e03-9728-bb863c717910 is the<br>
>> incremental backup (using 16 MiB).<br>
>><br>
>> Zitat von Rajat Dhasmana <<a href="mailto:rdhasman@redhat.com" target="_blank">rdhasman@redhat.com</a>>:<br>
>><br>
>> > Hi Satish,<br>
>> ><br>
>> > Did you check the size of the actual backup file in ceph storage? It<br>
>> should<br>
>> > be created in the *backups* pool[1].<br>
>> > Cinder shows the same size of incremental backup as a normal backup but<br>
>> > file size should be different from<br>
>> > the size shown in cinder DB records. Also file size of incremental backup<br>
>> > should not be the same as the file size of full backup.<br>
>> ><br>
>> > [1]<br>
>> ><br>
>> <a href="https://github.com/openstack/devstack/blob/34afa91fc9f830fc8e1fdc4d76e7aa6d4248eaaa/lib/cinder_backups/ceph#L22" rel="noreferrer" target="_blank">https://github.com/openstack/devstack/blob/34afa91fc9f830fc8e1fdc4d76e7aa6d4248eaaa/lib/cinder_backups/ceph#L22</a><br>
>> ><br>
>> > Thanks<br>
>> > Rajat Dhasmana<br>
>> ><br>
>> > On Wed, May 17, 2023 at 12:25 AM Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>><br>
>> wrote:<br>
>> ><br>
>> >> Folks,<br>
>> >><br>
>> >> I have ceph storage for my openstack and configure cinder-volume and<br>
>> >> cinder-backup service for my disaster solution. I am trying to use the<br>
>> >> cinder-backup incremental option to save storage space but somehow It<br>
>> >> doesn't work the way it should work.<br>
>> >><br>
>> >> Whenever I take incremental backup it shows a similar size of original<br>
>> >> volume. Technically It should be smaller. Question is does ceph<br>
>> >> support incremental backup with cinder?<br>
>> >><br>
>> >> I am running a Yoga release.<br>
>> >><br>
>> >> $ openstack volume list<br>
>> >><br>
>> +--------------------------------------+------------+------------+------+-------------------------------------+<br>
>> >> | ID | Name | Status |<br>
>> >> Size | Attached to |<br>
>> >><br>
>> +--------------------------------------+------------+------------+------+-------------------------------------+<br>
>> >> | 285a49a6-0e03-49e5-abf1-1c1efbfeb5f2 | spatel-vol | backing-up |<br>
>> >> 10 | Attached to spatel-foo on /dev/sdc |<br>
>> >><br>
>> +--------------------------------------+------------+------------+------+-------------------------------------+<br>
>> >><br>
>> >> ### Create full backup<br>
>> >> $ openstack volume backup create --name spatel-vol-backup spatel-vol<br>
>> --force<br>
>> >> +-------+--------------------------------------+<br>
>> >> | Field | Value |<br>
>> >> +-------+--------------------------------------+<br>
>> >> | id | 4351d9d3-85fa-4cd5-b21d-619b3385aefc |<br>
>> >> | name | spatel-vol-backup |<br>
>> >> +-------+--------------------------------------+<br>
>> >><br>
>> >> ### Create incremental<br>
>> >> $ openstack volume backup create --name spatel-vol-backup-1<br>
>> >> --incremental --force spatel-vol<br>
>> >> +-------+--------------------------------------+<br>
>> >> | Field | Value |<br>
>> >> +-------+--------------------------------------+<br>
>> >> | id | 294b58af-771b-4a9f-bb7b-c37a4f84d678 |<br>
>> >> | name | spatel-vol-backup-1 |<br>
>> >> +-------+--------------------------------------+<br>
>> >><br>
>> >> $ openstack volume backup list<br>
>> >><br>
>> +--------------------------------------+---------------------+-------------+-----------+------+<br>
>> >> | ID | Name |<br>
>> >> Description | Status | Size |<br>
>> >><br>
>> +--------------------------------------+---------------------+-------------+-----------+------+<br>
>> >> | 294b58af-771b-4a9f-bb7b-c37a4f84d678 | spatel-vol-backup-1 | None<br>
>> >> | available | 10 |<br>
>> >> | 4351d9d3-85fa-4cd5-b21d-619b3385aefc | spatel-vol-backup | None<br>
>> >> | available | 10 |<br>
>> >><br>
>> +--------------------------------------+---------------------+-------------+-----------+------+<br>
>> >><br>
>> >><br>
>> >> My incremental backup still shows 10G size which should be lower<br>
>> >> compared to the first backup.<br>
>> >><br>
>> >><br>
>> >><br>
>><br>
>><br>
>><br>
>><br>
>><br>
<br>
<br>
<br>
</blockquote></div>