Cloud storage

The HBP Cloud Service based at KIT provides users of the HBP Platforms with easy, secure and transparent access to Cloud-based storage and computing resources, complementing the storage provided locally by the HPC centres.

The Cloud-based storage with a capacity of 480 TB is accessible through an Amazon S3-compatible interface.

More information about the Cloud storage:
http://wiki.scc.kit.edu/lsdf/index.php/S3_on_WOS

The Cloud storage can be accessed using UNICORE:
http://wiki.scc.kit.edu/lsdf/index.php/UNICORE_and_S3

Technical documentation

Scripts for synchronizing the S3 accounts with the HBP LDAP and for the UNICORE mapping of accounts are developed and in use:

Version of software 1.0
Version of documentation 1.0
Date of release 13th July 2015
Software & documentation Available on demand (diana.gudu@kit.edu)
Requirements and dependencies
Deployment The scripts should be deployed on an infrastructure consisting of

  1. the server hosting the UNICORE/X service, and
  2. the WOS S3 user management node.
Target users HPC Platform infrastructure team
Responsible Diana Gudu (diana.gudu@kit.edu), T7.5.5
The KIT developed scripts for synchronising the local S3 accounts with the master LDAP server of the HBP. The scripts also maintain the UNICORE file mapping between users and S3 keys, which ultimately enables access to S3 storage through UNICORE. On one side, a REST API was developed for managing the S3 access keys, running on the S3 management node. On the other side, on the machine hosting the UNICORE services, a script regularly queries the LDAP server for new users and uses the REST API to create new keys and update the UNICORE user mapping. The scripts are designed for local use within the KIT cloud storage infrastructure.