Dynamic Storage System

Okt. 2022 - Nov. 2022

Java, Spring Boot, MySQL

Dynamic Storage System

The Dynamic Storage System is a scalable storage service REST API designed with Spring Boot to efficiently and securely store and retrieve files.

It is built on two layers: the user access layer and the file access layer. The user access layer is responsible for handling incoming user requests, managing access to the database and performing file encryption. It communicates with the file access layer, which acts as the actual storage service. Together, these two layers provide a dynamic storage system capable of securely store files on distributed systems.

Features

Buckets, Directories and Files

Support for common object storage structures used by AWS S3 or Azure Blob.
Scalable Service

Add new servers to increase storage and request capacities.
File Encryption

Files are encrypted with a user specific secret key.
Token based authentication

A Json Web Token is used to authenticate users.
Access Control (WIP)

Grant certain access permissions to specific users and buckets.
Redundancy (WIP)

The files can be stored redundantly to improve reliability.

API Structure

Authentication

Except for user creation, you have to provide an authentication bearer token in every request inside of the Authorization header. The user-specific token is generated during user creation and should be kept secret (Refresh tokens are not implemented yet).

User Creation

/api/users/ POST PUT

          
{
  "firstname": "John",
  "lastname": "Doe",
  "email": "john@doe.com"
}

When creating a new user, please provide an email address, the user's first and last name inside a json object. When updating a user using the PUT-Method, please provide the authentication token inside the Authorization header.

Response 200

          
{
  "status": "ok",
  "userId": 1,
  "token": "SECRET TOKEN"
}

User Fetching

/api/users/{user_id} GET

Currently, only the user that sends the request can fetch his own data (who am i). When fetching a user, please provide the user id inside the url and the authentication token inside the Authorization header.

Response 200

          
{
  "id": {user_id},
  "firstname": "John",
  "lastname": "Doe",
  "email": "john@doe.com"
}

User Deletion

/api/users/{user_id} DELETE

When deleting a user, please provide the user id inside the url and the authentication token inside the Authorization header.

Response 200

          
{
  "status": "ok",
  "message": "User deleted"
}

Bucket Creation

/api/buckets/?bucket=NAME POST

When creating a new bucket, please provide the bucket name inside the `bucket` query parameter and the authentication token inside the Authorization header.

Response 200

          
{
  "status": "ok",
  "message": "Bucket successfully created."
}

Bucket Listing

/api/buckets/ GET

List all buckets that belong to the user that sends the request. When listing all buckets, please provide the authentication token inside the Authorization header.

Response 200

          
{
  "status": "ok",
  "message": "Buckets successfully loaded.",
  "data": [
    {
      "id": 1,
      "name": "BUCKET_NAME",
      "folders": 0,
      "files": 0,
      "users": 0
    }
  ],
  "count": 1,
  "total": 1,
  "page": 0,
  "pages": 1
}

Bucket Deletion

/api/buckets/{bucket_name} DELETE

Delete a bucket that belongs to the user that sends the request. Please provide the bucket name inside the url and the authentication token inside the

Response 200

          
{
  "status": "ok",
  "message": "Bucket successfully deleted."
}

Bucket Fetching

/api/buckets/{bucket_name} GET

Show information about the bucket that is requested. Please provide the bucket name inside the url and the authentication token inside the Authorization header.

Response 200

          
{
  "status": "ok",
  "message": "Bucket info successfully loaded.",
  "data": {
    "id": 4,
    "name": "BUCKET_NAME",
    "folders": 1,
    "files": 1,
    "users": 0
  }
}

Folder Creation

/api/folders/{bucket}/{parents}?folder=NAME GET

            
Example 1: /api/folders/my_bucket/parent1%2Fparent2?folder=my_folder
Example 2: /api/folders/my_bucket?folder=my_folder

Create a new folder inside a bucket or another folder. Please note that `parents` is url encoded and that sub-folders in `parents` are separated with %2F (encoding of /). The PathVariable `parents` is optional. Not providing it will create the folder inside the buckets root folder. Please provide the bucket name, any parents if given, and the folder name inside the url and the authentication token inside the Authorization header.

Response 200

          
{
  "status": "ok",
  "message": "Bucket info successfully loaded.",
  "data": {
    "id": 4,
    "name": "BUCKET_NAME",
    "folders": 1,
    "files": 1,
    "users": 0
  }
}

Folder Listing

/api/folders/{bucket}/{parents}/ GET

List all folders inside a bucket or another folder. Please note that `parents` is url encoded and that sub-folders in `parents` are separated with %2F (encoding of /). The PathVariable `parents` is optional. Not providing it will list the folders inside the buckets root folder. Please provide the bucket name, any parents if given inside the url and the authentication.

Response 200

          
{
  "status": "ok",
  "message": "Storage items successfully loaded.",
  "data": [
    {
      "id": 2,
      "name": "FOLDER_NAME",
      "path": "/BUCKET_NAME/FOLDER_PATH",
      "folders": 0,
      "files": 0
    }
  ],
  "count": 1,
  "total": 1,
  "page": 0,
  "pages": 1
}

Folder Deletion (Not implemented yet)

/api/folders/{bucket}/{folder_path} DELETE

(Not implemented yet)
Delete a folder inside a bucket or another folder. Please note that `folder_path` is url encoded and that sub-folders in `folder_path` are separated with %2F (encoding of /). Please provide the bucket name, the folder path inside the url and the authentication token inside the Authorization header.

Response 200

          
{
  "status": "ok",
  "message": "Folder successfully deleted.",
}

File Upload

/api/files/{bucket}/{folder_path}/ POST

Upload a file inside a bucket or a folder. Please note that `folder_path` is url encoded and that sub-folders in `folder_path` are separated with %2F (encoding of /). Please provide the bucket name, the folder path inside the url and the authentication token inside the Authorization header. The file must be provided as a multipart/form-data request.

Response 200

          
{
  "path": "/BUCKET_NAME/FOLDER_PATH/FILE_NAME",
  "status": "ok",
  "message": "File stored."
}

File Listing

/api/files/{bucket}/{folder_path}/ GET

List all files inside a bucket or a folder. Please note that `folder_path` is url encoded and that sub-folders in `folder_path` are separated with %2F (encoding of /). Please provide the bucket name, the folder path inside the url and the authentication token inside the Authorization header.

Response 200

          
{
  "status": "ok",
  "message": "Storage items successfully loaded.",
  "data": [
    {
      "id": 1,
      "name": "FILE_NAME",
      "path": "/BUCKET_NAME/FOLDER_PATH/FILE_NAME",
      "file_type": "FILE_TYPE",
      "size": FILE_SIZE
    },
  ],
  "count": 1,
  "total": 1,
  "page": 0,
  "pages": 1
}

File Download

/api/files/download/{bucket}/{file_path}/ GET

Download a file inside a bucket or a folder. Please note that `file_path` is url encoded and that sub-folders in `file_path` are separated with %2F (encoding of /). Please provide the bucket name, the file path inside the url and the authentication token inside the Authorization header.

Response 200

          
< File >

File Download (Not implemented yet)

/api/files/{bucket}/{file_path}/ DELETE

(Not implemented yet)
Delete a file inside a bucket or a folder. Please note that `file_path` is url encoded and that sub-folders in `file_path` are separated with %2F (encoding of /). Please provide the bucket name, the file path inside the url and the authentication token inside the Authorization header.

Response 200

          
{
  "status": "ok",
  "message": "File successfully deleted.",
}