Skip to main content

Sync Backup Plans Introduction

What is a Sync Backup Plan?​

A Sync Backup Plan in Pluton is a real-time or scheduled synchronization task that keeps a destination storage in sync with your source files and folders. Unlike traditional incremental backups that create snapshots, sync plans maintain a mirror of your data where the destination matches the source at all times.

Pluton uses Rclone under the hood - a powerful cloud synchronization tool that efficiently transfers and syncs files to and from various storage providers. Sync plans are ideal when you need live copies of your data rather than versioned snapshots.

Key Differences: Sync vs. Incremental Backup​

Sync Backup Plans (Rclone)​

  • Mirror synchronization - Destination mirrors the source
  • Latest version only - No snapshot history by default
  • Real-time or frequent updates - Can run every few minutes
  • Bi-directional capable - Can sync in both directions
  • Fast updates - Only changed files are transferred
  • Simple recovery - Files are directly accessible at destination
  • Optional revision history - Can maintain file versions in a history folder

Incremental Backup Plans (Restic)​

  • Snapshot-based - Creates point-in-time snapshots
  • Version history - Keep multiple snapshots over time
  • Periodic backups - Typically hourly, daily, or weekly
  • One-way only - Source to destination
  • Deduplication - Efficient storage of unchanged data
  • Restore process - Requires restoration from snapshots
  • Built-in retention - Automatic pruning of old snapshots

When to Use Each​

Use Sync Backup Plans when:

  • You need a live, up-to-date copy of your data
  • You want files directly accessible without restoration
  • You're mirroring data to another location
  • You need frequent updates (every few minutes)
  • You want bi-directional synchronization
  • You're maintaining off-site copies of active data

Use Incremental Backup Plans when:

  • You need version history and point-in-time recovery
  • You want to restore files to a previous state
  • You need efficient long-term archival
  • You're protecting against accidental deletion or corruption
  • You want automatic retention policies
  • Storage space efficiency is important

Key Features​

🔄 Real-Time Synchronization​

  • Frequent updates - Run syncs every few minutes
  • Immediate propagation - Changes reflected quickly at destination
  • Continuous monitoring - Keep destinations up-to-date
  • Efficient transfers - Only modified files are synced

📅 Flexible Scheduling​

  • High-frequency intervals - Every 5, 10, 15 minutes, or custom
  • Traditional schedules - Hourly, daily, weekly if preferred
  • Precise timing - Set specific times for sync execution
  • Custom intervals - Configure any frequency that suits your needs

🔄 Bi-Directional Sync (Optional)​

  • Two-way synchronization - Keep both locations in sync
  • Conflict resolution - Newer files take precedence
  • Automatic merging - Changes from both sides are combined
  • Safe synchronization - Prevents data loss during conflicts

📁 Change Tracking​

  • Added files - New files copied to destination
  • Modified files - Updated files overwritten at destination
  • Deleted files - Removed from destination to match source
  • Detailed logs - Track every change made during sync

🔒 Security & Encryption​

  • In-transit encryption - Secure transfers to cloud storage
  • At-rest encryption - Optional encryption at destination
  • Secure protocols - HTTPS, SFTP, and encrypted connections
  • Authentication - Secure access to storage providers

📦 Optional Revision History​

  • File versions - Keep previous versions in a history folder
  • Configurable retention - Control how many revisions to keep
  • Protection against deletions - Recover accidentally deleted files
  • Balance between sync and backup - Get benefits of both approaches

🎯 Selective Synchronization​

  • Include paths - Specify exactly which folders/files to sync
  • Exclude patterns - Skip temporary files, caches, or specific patterns
  • Flexible filtering - Fine-grained control over what gets synced

🔔 Notifications​

  • Email notifications - Get notified on sync failures
  • Webhook integration - Integrate with external monitoring systems
  • Status updates - Stay informed about sync operations

🔁 Retry Mechanism​

  • Automatic retries - Failed syncs are automatically retried
  • Configurable attempts - Set the number of retry attempts and delay
  • Network resilience - Handles temporary connectivity issues

📊 Progress Tracking​

  • Real-time statistics - Monitor files added, modified, removed
  • Transfer metrics - View data transferred and sync speed
  • Change history - Review what changed in each sync operation
  • Detailed logs - Complete logs of all sync operations

🌐 Multi-Device Support​

  • Local sync - Sync from the main Pluton server
  • Remote sync - Sync from remote devices via Pluton Agent
  • Centralized management - Manage all sync operations from one interface

How It Works​

Sync Process Flow​

Technical Process​

  1. Initialization

    • When you create a sync plan, Pluton verifies connectivity to the destination
    • The destination path is created if it doesn't exist
    • A sync schedule is registered based on your interval settings
    • Optional history folder is created for revision tracking

  2. Scheduled Execution

    • At the scheduled time, the cron manager triggers the sync task
    • The task is added to a priority queue for execution
    • Supports configurable retries (default: 5 attempts) with delays between retries

  3. Sync Execution

    • Pluton reads the source paths from your configuration
    • Rclone compares source files with destination files
    • New or modified files are identified by size, modification time, or checksum
    • Changes are transferred to the destination

  4. Change Application

    • Added files - Copied to destination
    • Modified files - Updated at destination
    • Deleted files - Removed from destination (if configured)
    • All operations maintain file attributes (permissions, timestamps)

  5. Revision History (Optional)

    • Before overwriting or deleting files, old versions are moved to history folder
    • Revision cleanup happens based on retention policy
    • Protects against accidental overwrites or deletions

  6. Metadata Recording

    • Sync statistics are recorded (files changed, data transferred)
    • Change history is stored for review
    • Logs are updated with detailed operation information

  7. Notifications & Logging

    • Sync completion status is logged
    • Notifications are sent on failures (configurable)
    • Progress and statistics are updated in the database

Sync Strategies​

Standard Sync (One-Way)​

  • Source files are copied/updated to destination
  • Deletions at source are reflected at destination
  • Destination becomes exact mirror of source
  • Use case: Backing up active data, offsite copies

Sync with Revisions​

  • Previous file versions saved before overwriting
  • Deleted files moved to history folder
  • Configurable retention for revisions
  • Use case: Protection against accidental changes while maintaining sync

Copy-Only Mode​

  • Only copies new files, doesn't delete at destination
  • Modified files are updated
  • Deleted files remain at destination
  • Use case: Accumulating copies, one-way replication without deletions

Storage Destinations​

Sync backup plans support the same storage types as incremental backups:

  • Local Storage - Sync to local disks or mounted drives
  • Cloud Storage - AWS S3, Google Cloud Storage, Azure Blob
  • Object Storage - Backblaze B2, Wasabi, MinIO
  • Cloud Providers - Google Drive, OneDrive, Dropbox, pCloud
  • SFTP/SSH - Any server with SSH access
  • Network Shares - SMB/CIFS, NFS mounts
  • And more - Any storage supported by Rclone (75+ backends)

Performance Optimization​

Pluton provides advanced performance settings specifically for sync operations:

  • Transfers - Number of parallel file transfers (default: 4)
  • Buffer Size - Memory buffer for streaming large files (default: 16M)
  • Multi-thread Streams - Concurrent streams for large files (default: 4)
  • Checksum Strategy - Speed vs. accuracy tradeoff
    • Default - Compare by size and modtime
    • Checksum - Compare using checksums (slower but accurate)
    • Size only - Compare only by file size (fastest)
  • Max Duration - Limit how long a sync can run
  • Max Transfer - Limit total data transferred per sync

Use Cases​

Active Data Mirroring​

  • Keep a live copy of your working files in the cloud
  • Maintain synchronized folders across multiple locations
  • Replicate data to backup servers in real-time

Offsite Data Protection​

  • Continuously sync critical data to offsite locations
  • Maintain up-to-date disaster recovery copies
  • Quick recovery without restoration process

Collaboration & Sharing​

  • Sync shared folders to cloud storage for team access
  • Maintain synchronized project files across devices
  • Keep documentation or resources up-to-date

Media & Asset Management​

  • Sync photos/videos as they're created
  • Maintain libraries across multiple storage locations
  • Keep media collections synchronized

Development & CI/CD​

  • Sync build artifacts to storage
  • Keep deployment packages updated
  • Synchronize configuration files across environments

Data Distribution​

  • Distribute files to multiple locations simultaneously
  • Keep edge servers synchronized with central data
  • Replicate content to CDN origins

Best Practices​

  1. Choose the Right Frequency - Balance between freshness and performance impact

    • High-priority data: Every 5-15 minutes
    • Normal data: Hourly or daily
    • Archives: Weekly or as needed

  2. Use Exclude Patterns - Don't sync unnecessary files

    • Temporary files (*.tmp, *.temp)
    • System files (.DS_Store, Thumbs.db)
    • Build artifacts (node_modules/, target/)
    • Cache directories

  3. Enable Revisions for Important Data - Protect against accidents

    • Enable for user-generated content
    • Set appropriate retention (e.g., 30 days)
    • Disable for easily reproducible data

  4. Monitor Sync Performance - Keep syncs efficient

    • Review sync durations regularly
    • Adjust transfer settings if needed
    • Check for consistently failed syncs

  5. Test Before Production - Verify sync behavior

    • Start with a small test folder
    • Verify deletions work as expected
    • Confirm revisions are created if enabled

  6. Use Tags for Organization - Group related sync plans

    • Tag by data type (e.g., "media", "documents")
    • Tag by priority level
    • Tag by department or project

  7. Configure Notifications - Stay informed

    • Enable email notifications for failures
    • Set up webhooks for monitoring integration
    • Review logs regularly

  8. Optimize for Your Storage - Adjust settings per destination

    • Use checksums for critical data
    • Increase transfers for fast storage
    • Tune buffers for network conditions

Sync vs. Backup: Making the Choice​

Use Sync Plans For:​

✅ Active, frequently changing data
✅ Need immediate access to files
✅ Mirroring data across locations
✅ High-frequency updates (minutes)
✅ Simple file-level recovery
✅ Live collaboration scenarios

Use Incremental Backups For:​

✅ Long-term data protection
✅ Point-in-time recovery needs
✅ Compliance and archival requirements
✅ Efficient storage with deduplication
✅ Protection against corruption over time
✅ Scheduled periodic protection

Use Both Together:​

Many users benefit from combining both approaches:

  • Sync plans for active data that needs real-time protection
  • Incremental backups for the same data for long-term history
  • Example: Sync documents hourly + backup them daily with retention

Next Steps​

For technical details about Rclone, visit the official documentation: Rclone