JarCasting
JarCasting JarCasting

filesystem

The file system library will help perform operations on AWS S3 or SFTP file system. The library will be useful for big data projects where they need to perform operations on files.

License

License
GroupId

GroupId

com.github.ramalapure
ArtifactId

ArtifactId

filesystem
Last Version

Last Version

1.1
Release Date

Release Date
Type

Type

jar
Description

Description

filesystem
The file system library will help perform operations on AWS S3 or SFTP file system. The library will be useful for big data projects where they need to perform operations on files.
Project URL

Project URL

https://github.com/RamAlapure/filesystem
Source Code Management

Source Code Management

https://github.com/RamAlapure/filesystem

Download filesystem

Filename Size
filesystem-1.1.pom
filesystem-1.1.jar 20 KB
filesystem-1.1-sources.jar 14 KB
filesystem-1.1-javadoc.jar 84 KB
Browse

How to add to project

<!-- https://jarcasting.com/artifacts/com.github.ramalapure/filesystem/ -->
<dependency>
    <groupId>com.github.ramalapure</groupId>
    <artifactId>filesystem</artifactId>
    <version>1.1</version>
</dependency>
// https://jarcasting.com/artifacts/com.github.ramalapure/filesystem/
implementation 'com.github.ramalapure:filesystem:1.1'
// https://jarcasting.com/artifacts/com.github.ramalapure/filesystem/
implementation ("com.github.ramalapure:filesystem:1.1")
'com.github.ramalapure:filesystem:jar:1.1'
<dependency org="com.github.ramalapure" name="filesystem" rev="1.1">
  <artifact name="filesystem" type="jar" />
</dependency>
@Grapes(
@Grab(group='com.github.ramalapure', module='filesystem', version='1.1')
)
libraryDependencies += "com.github.ramalapure" % "filesystem" % "1.1"
[com.github.ramalapure/filesystem "1.1"]

Dependencies

runtime (5)

Group / Artifact Type Version
org.apache.commons : commons-io jar 1.3.2
org.reflections : reflections jar 0.9.11
org.projectlombok : lombok jar 1.18.8
com.amazonaws : aws-java-sdk-s3 jar 1.11.579
com.jcraft : jsch jar 0.1.55

Project Modules

There are no modules declared in this project.

The library cab be used to perform file system operations, currently it support AWS S3 and SFTP related operations.

How you can use it:

Maven Central

Maven

<dependency>
  <groupId>com.github.ramalapure</groupId>
  <artifactId>filesystem</artifactId>
  <version>1.1</version>
</dependency>

Gradle

implementation 'com.github.ramalapure:filesystem:1.1'

Let's see the configuration and how to get the instance of file system.

for AWS S3 file system:

Map<String, Object> properties = new HashMap<>();
properties.put(AppConstants.S3_ACCESS_KEY, "REPLACE_AWS_ACCESS_KEY");
properties.put(AppConstants.S3_SECRET_KEY, "REPLACE_AWS_SECRET_KEY");
properties.put(AppConstants.REGION, "REPLACE_REGION");
properties.put(AppConstants.S3_BUCKET_NAME, "REPLACE_BUCKET_NAME");
Configuration config = new Configuration(AppConstants.STR_S3, properties);
FileSystem fileSystem = FileSystemFactory.getFileSystem(config);

for SFTP file system:

Map<String, Object> properties = new HashMap<>();
properties.put(AppConstants.SFTP_USERNAME, "REPLACE_SFTP_USERNAME");
properties.put(AppConstants.SFTP_PASSWORD, "REPLACE_SFTP_PSWD");
properties.put(AppConstants.SFTP_HOSTNAME, "REPLACE_SFTP_HOST"); 
// port value must be integer
properties.put(AppConstants.SFTP_PORT, REPLACE_SFTP_PORT);
Configuration config = new Configuration(AppConstants.STR_SFTP, properties);
FileSystem fileSystem = FileSystemFactory.getFileSystem(config);

The AWS S3 file system supports following operations on bucket:

  1. Create new bucket 
    // As creating a bucket is not directly available in file system interface 
// we need to get the actual instance of S3 client.
AwsS3Client client = (AwsS3Client) fileSystem;
client.createBucket("REPLACE_WITH_BUCKET_NAME");
  2. Get list of buckets 
    // As list of buckets is not directly available in file system interface 
// we need to get the actual instance of S3 client.
AwsS3Client client = (AwsS3Client) fileSystem;
List<String> listOfBuckets = client.getListOfBuckets();
  3. Get list of files from specific folder or bucket 
    // Empty string will return all the objects/files from buckets
List<String> files = fileSystem.getListOfFiles("");
//OR you can get files from only specific folder
List<String> files = fileSystem.getListOfFiles("/specific-folder");
  4. Upload file/object to bucket 
    File file = new File("THE_FILE_YOU_WANT_TO_UPLOAD");
fileSystem.uploadFile(new FileInputStream(file), "FILE_KEY");
  5. Read input stream of file from bucket 
    InputStream inputStream = fileSystem.read("FILE_KEY");
  6. Download the file from bucket to local system 
    fileSystem.downloadFile("FILE_KEY_TO_DOWNLOAD_FROM_BUCKET", "FILE_NAME_TO_SAVE_ON_LOCAL_SYSTEM");
  7. Create folder in bucket 
    fileSystem.createFolder("FOLDER_NAME");
  8. Delete file from bucket 
    fileSystem.deleteFile("FILE_KEY");
  9. Copy file/object from one folder to another in same bucket 
    fileSystem.copyFile("SOURCE_FILE_KEY", "TARGET_FILE_KEY");
  10. Delete the bucket with objects and versions 
    // As deleting a bucket is not directly available in file system interface 
// we need to get the actual instance of S3 client.
AwsS3Client client = (AwsS3Client) fileSystem;
client.deleteBucket("BUCKET_NAME");

The SFTP file system supports following operations on directory/file:

  1. Get list of files 
    // Empty string will return all the files
List<String> files = fileSystem.getListOfFiles("");
//OR you can get files from only specific folder
List<String> files = fileSystem.getListOfFiles("/specific-folder");
  2. Upload file 
    File file = new File("THE_FILE_YOU_WANT_TO_UPLOAD");
fileSystem.uploadFile(new FileInputStream(file), "FILE_KEY");
  3. Read input stream of file 
    InputStream inputStream = fileSystem.read("FILE_NAME_WITH_PATH");
  4. Download file to local system 
    fileSystem.downloadFile("FILE_NAME_TO_DOWNLOAD_FROM_DIRECTORY", "FILE_NAME_TO_SAVE_ON_LOCAL_SYSTEM");
  5. Create new folder 
    fileSystem.createFolder("FOLDER_NAME");
  6. Delete file/directory 
    fileSystem.deleteFile("FILE_NAME_WITH_PATH");
  7. Copy file from one folder to another 
    fileSystem.copyFile("SOURCE_FILE_NAME_WITH_PATH", "TARGET_FILE_NAME_WITH_PATH");

Versions

Version
1.1
1.0