Jetbackup 5 API

[gregh]

Hi guys,

I'm looking at rolling out Jetbackup onto a Cpanel fleet for customer self service backups using an S3-like storage target.

Because of the large number of servers involved I'd really like to use an API to do initial setup / destination config / schedule config and so on, but I don't see any mention of an API in the Jetbackup 5 documentation and some best guesses based on the JB4 api reference just gets me a "No module provided" message from the jetapi command line tool.

Questions:

Does JB5 have an API interface, if so can you point me to the documentation on interacting with it?

Do you have a roadmap for the S3 storage plugin becoming production ready rather than Experimental?

If JB5 does not have an API interface, how long is JB4 going to remain supported?

If the better bet for group server management is the JetBackup Management console, what is the roadmap for that becoming a production ready product without the big Alpha warnings in the documentation?

Thanks,
Greg.

[JetAppsRichard]

Hello Greg,

Apologies for the delay and thank you for your inquiry.

Our API Documentation for the JetBackup 5 is available via direct link: https://docs.jetbackup.com/v5.1/api/index.html
Please note that unfortunately, the JB5 API documentation is not fully complete but we will continue to improve and add to this in the coming weeks/months. You can find a list of all jetbackup5api commands by running the following command in your terminal as the root user:

Code: Select all

jetbackup5api -F

You will find that many of the commands are very similar to the JetBackup 4 API. Feel free to use the JetBackup 4 API Docs for reference when assigning the correct parameters. If you have any questions regarding a specific command and how to use it, please let us know!

Do you have a roadmap for the S3 storage plugin becoming production ready rather than Experimental?

Currently, the S3 Destination Plugin is experimental on a separate track to stable within 3-6 months. We would advise following our social media channels and JetApps.com blog for information on when the S3 Plugin is moved from Experimental to Stable.

If the better bet for group server management is the JetBackup Management console, what is the roadmap for that becoming a production ready product without the big Alpha warnings in the documentation?

The Management Console currently only supports JetBackup 4 for cPanel and JetBackup 4 for Linux, and currently does not support JetBackup 5. Our current estimate for a new JetBackup MC version with full support for JetBackup 5 is in 2022, but we are looking into the feasibility of backporting JB5 support to the current JBMC version.

We do not currently have an ETA for stable Management Console as we are currently focusing our development resources on JetBackup 5. With that said, many clients have been happy with the stability of the Management console - it is in use for production environments in its current state and is in alpha as we're planning more features/bug fixes to add to it. It is recommended to try it with JetBackup 4 if you feel you would benefit from using it!

Please let us know if you have any other questions or concerns!

[gregh]

Hi Richard,

Thanks for getting back to me.

I've managed to automate most of what I was looking for around the install and configure, I now have some more questions around the S3 plugin.

We would like to write directly to the Infrequent Access tier on S3 storage for the obvious savings across massive data sets, does the S3 destination plugin support tagging the files to write directly to S3 Infrequent Access (ie. corresponding to the --storage-class attribute on the aws cli command "aws s3 cp") ? if not is that feature on the road map?

Is there any plans to enable incremental backups to be archived / compressed to get around the 128k minimum file size for the Infrequent Access storage tier?

Also, are there any plans to allow scheduling to take one of the weekly backups and convert it into a "monthly" for longer retention?

oh, and I'm aware the API docs you've linked me to are incomplete, but would you like feedback on the parts that are there or should I wait til you guys are actually happy with them first?

Thanks,
Greg.

[JetAppsRichard]

Hello Greg,

Glad to see you were able to utilize our JetBackup 5 API for your automation. With regards to your new queries:

does the S3 destination plugin support tagging the files to write directly to S3 Infrequent Access (ie. corresponding to the --storage-class attribute on the aws cli command "aws s3 cp") ? if not is that feature on the road map?

The JetBackup 5 S3 Compatible Plugin now utilizes Amazon Simple Storage (S3) API rather than the AWS CLI to perform backups and restores. Our developers have decided to do this in order to allow for support for other S3 Compatible vendors such as Wasabi, BackBlaze, and ScaleWay! With this being the case, there is no current feature that will allow custom flags to be attached to an object to write directly to the S3 Infrequent Access. I would highly recommend opening a feature request at our JetApps Request Site in order to gain votes on this feature. Our developers utilize the voting feature to gauge the demand for certain features and are actively checking the request site to find new popular features to add to our development map.

Is there any plans to enable incremental backups to be archived / compressed to get around the 128k minimum file size for the Infrequent Access storage tier?

Unfortunately, JetBackup 5 is not able to change the backup structure of backups in the destination that have already been generated. Downloading an incremental backup to your server will automatically compress the data into a .tar.gz file that you could then use to upload manually to the infrequent access storage tier.

are there any plans to allow scheduling to take one of the weekly backups and convert it into a "monthly" for longer retention?

JetBackup 5 has a backup lock feature that will allow you to set a custom TTL for the backup past its retention. You can lock multiple different account backups from the Restore and Download page or by lock a single account backup through the View Backups page.

would you like feedback on the parts that are there or should I wait til you guys are actually happy with them first?

We would be more than happy to hear your feedback regarding our API Documentation. As we are actively developing the docs we will be able to incorporate your feedback much quicker. Thank you for your continued support!

[gregh]

Thanks for the update Richard,

Now some more questions about the backup structure

How does the rotation of backup jobs work when the oldest incremental with the initial diff rolls off? Are those files moved to the new oldest incremental, is there a logical operation, the contents stay the same on S3 and the backup list of where files are is updated etc? This would affect lifecycle policies moving the files to Infrequent access.

Do we need regular full backups in the schedule to base the incremental off of?

How does the restore roll the incremental together to do a point in time restore for a customer?

For context we are in initial testing and development before rolling Jetbackup out to our customer base, due to the quantity of data we are sensitive to the ways that S3 is getting used by the backup application (those S3 costs add up quickly!). If there is a best practise way to structure the backups to minimise S3 usage costs we are all ears, as the solution has not yet been rolled out to customers so we can change anything we need to at this stage.


For documentation feedback,

https://docs.jetbackup.com/v5.1/api/Bac ... pjobs.html

In the Parameter dictionary "Type" is listed as having "Incremental" and "Directories" as it's possible settings, I think that is meant to be "Accounts" and "Directories".

[JetAppsRichard]

Hello Greg,

Are those files moved to the new oldest incremental, is there a logical operation, the contents stay the same on S3 and the backup list of where files are is updated etc? How does the restore roll the incremental together to do a point in time restore for a customer?

That is correct, once the initial backup has reached the retention, it is replaced by the next snap. JetBackup 5's new indexing engine will keep track of the contents stored in the s3 destination and will properly gather the data needed for a point in time restore. Currently, we do not know how this will fully affect lifecycle policies as this would require further tests. As you may know, the S3 Compatible Plugin is still currently tagged as Experimental as we continue to test its limitations and continue stabilizing its functionality.

Do we need regular full backups in the schedule to base the incremental off of?

Yes, it is required that the full backups remain in the schedule in order to properly continue incremental backups. If the incremental pushes ever fall out of sync, running a reindex on the destination could help sync JetBackup with the existing backups on the destination. You can see how to reindex a destination by checking here: Reindex Destination

If there is a best practice way to structure the backups to minimise S3 usage costs we are all ears

S3 Incremental backups with JetBackup 5 are optimized to minimize the cost of storage with your S3 Compatible vendor. Although we don't have any official "best practices", we have seen reports of users lowering their costs by utilizing other S3 Compatible Vendors such as Wasabi.

Lastly, thank you for the feedback regarding the API documentation. I have marked the inconsistency and it will be fixed as we begin to push updates to the live documentation pages.