Skip to content
/ s3sync Public
forked from seqsense/s3sync

♻️ Golang utility for syncing between s3 and local, similar to `aws s3 sync`

License

Notifications You must be signed in to change notification settings

Gompei/s3sync

 
 

Repository files navigation

s3sync

CI codecov

Golang utility for syncing between s3 and local

Usage

Use New to create a manager, and Sync function syncs between s3 and local filesystem.

import (
  "github.com/aws/aws-sdk-go/aws"
  "github.com/aws/aws-sdk-go/aws/session"
  "github.com/seqsense/s3sync"
)

func main() {
  // Creates an AWS session
  sess, _ := session.NewSession(&aws.Config{
    Region: aws.String("us-east-1"),
  })

  syncManager := s3sync.New(sess)

  // Sync from s3 to local
  syncManager.Sync("s3://yourbucket/path/to/dir", "local/path/to/dir")

  // Sync from local to s3
  syncManager.Sync("local/path/to/dir", "s3://yourbucket/path/to/dir")
}
  • Note: Sync from s3 to s3 is not implemented yet.

Sets the custom logger

You can set your custom logger.

import "github.com/seqsense/s3sync"

...
s3sync.SetLogger(&CustomLogger{})
...

The logger needs to implement Log and Logf methods. See the godoc for details.

Sets up the parallelism

You can configure the number of parallel jobs for sync. Default is 16.

s3sync.new(sess, s3sync.WithParallel(16)) // This is the same as default.
s3sync.new(sess, s3sync.WithParallel(1)) // You can sync one by one.

License

Apache 2.0 License. See LICENSE.

About

♻️ Golang utility for syncing between s3 and local, similar to `aws s3 sync`

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Go 95.1%
  • Makefile 4.9%