Jetbackup with S3 compatible, structure file

[dolphin]

I have a question regarding how JetBackup handles deletions when storing backups in buckets. The current structure which i see in my bucket is as follows:

host120..vn/.jetbackup
host120..vn/jetbackup_1_1_665d9b7ee07e72d63b078122/_667f7567edd7a8a6020fd674/exports/full
host120.vn/jetbackup_1_1_665d9b7ee07e72d63b078122/_667f7567edd7a8a6020fd674/exports/meta
host120..vn/jetbackup_1_1_665d9b7ee07e72d63b078122/667f7567edd7a8a6020fd674/exports/snap.10
host120..vn/jetbackup_1_1_665d9b7ee07e72d63b078122/_667f7567edd7a8a6020fd674/files0000000001/668f98d1000c0918000000a4

I noticed that when JetBackup performs a DELETE operation, for example

Aug 8 10:04:43 0/0/0/393/393 204 152 - - ---- 2/1/0/0/0 0/0 "DELETE /host120.vn/host120.vn/jetbackup_1_1_665d9b7ee07e72d63b078122/_65aeb799c6e6c82792059eec/jetbackup.index HTTP/1.1" and snap.x


it seems to delete only the files with jetbackup.index and snap.x in their names. However, I'm wondering about the other objects. Are they not being deleted, or is there a different process for handling these files?

The image i get from: https://docs.jetbackup.com/v5.3/adminpa ... pe/S3.html

[JetAppsTrenton]

Hello dolphin,

To clarify, JetBackup handles backup deletion by marking the backups that have exceeded the retention rate and removing the associated files. These files would be cleaned and deleted during the Backup Clean-up and Integrity Check process.

Please note that the files marked for deletion will not be removed at the end of the backup job and will remain in the destination until the Backup Cleanup and Integrity Check process executes. You may change the frequency of the Integrity Check schedule to minimize the amount of disk space used.

You may find more information regarding this in our documentation:
https://docs.jetbackup.com/v5.3/adminpa ... /Retention

Trenton V
JetApps, LLC.
JetApps.com | JetBackup.com

[dolphin]

Hi sir,
Thank you for the helpful information.
I have one more question: If backups are marked as "Damaged," will they be deleted by the Backup Cleanup and Integrity Check process, or will they not be removed by any means?

[JetAppsTrenton]

Hello Dolphin,

The "Damaged" backups will remain at the destination and follow the standard retention policy schedule. You may still be able to restore items from the damaged backups using the WHM admin interface.

We do not have a feature to remove only the "Damaged" backups, however, you may use our `deleteAccountSnapshots` API call to remove a snapshot within a specific time range. Please note that you MUST upgrade to v5.3.10 EDGE to specify a time range for a snapshot deletion. Using this API call on v5.3.9 will delete ALL snapshots for a specified user regardless of destination or schedule.

Below is an example API call that will delete all snapshots generated between March 15, 2024, at 00:00 - May 13, 2024, at 00:00 (Server Time).

Code: Select all

# jetbackup5api -F deleteAccountSnapshots -D "user={USER_ID}&range_start=2024-03-15T00:00&range_end=2024-05-13T00:00&approve=0"

Once the snapshots have been marked for deletion, restart the service to immediately start snapshot cleanup:

Code: Select all

service jetbackup5d restart

https://docs.jetbackup.com/v5.3-EDGE/ap ... n%20allows

Trenton V
JetApps, LLC.
JetApps.com | JetBackup.com

[dolphin]

Hi sir,
Ok thank you, i am clear.

[dolphin]

Hi sir
I would like to understand how JetBackup manages file segmentation when backing up to an S3 destination. Specifically, I am interested in knowing when a backup for an account might result in files being split from files0000000001 into files0000000002.

Is there a rule or configuration that determines when this segmentation occurs? For example, is it based on file size limits or other factors related to the S3 storage destination?

[JetAppsTrenton]

Hello Dolphin,

Thank you for your patience!

The S3 destination uses our Jetindex engine to format and store backups. The new storage format applies JetBackup's new Proprietary backup structure allowing for more efficient backups and enhancing incremental backups.

Below is an example of the backup structure.

Code: Select all

.
├── export
│   ├── snap.1
│   ├── snap.2
│    ....
├── files
│   ├── 6258c0a30008839f00001146
│   ├── 6258c0a30009115600001148
│   ├── 6258c0a3000914d000001149
│   ├── 6258c0a30009180b0000114b
│   ├── 6258c0a300091a820000114c
│   ├── 6258c0a300091f3400001151
│    ....
├── snap.1
│   └── jetbackup.index
├── snap.2
│   └── jetbackup.index

The "files" directory is where the backup data is stored across all snapshots, and each object represents a file in the backup. Additionally, there will be a corresponding "snap" and "export" data which stores the information required for JetBackup to read this backup.

Kindly refer to our documentation below for more information:
https://docs.jetbackup.com/v5.3/adminpa ... comparison

Trenton V
JetApps, LLC.
JetApps.com | JetBackup.com

[dolphin]

Hi sir,

Thank you for providing more detailed information about the architecture of JetBackup with S3.

Regarding the question, "when a backup for an account might result in files being split from files0000000001 into files0000000002," I noticed that it occurs when the limit of 26,937 objects is reached.

In summary, the information on JetBackup with S3-compatible structure and the Backup Clean-up and Integrity Check process has been very useful, and I am quite satisfied with how I’ve applied it to my work.