From 73a8a4cde94ddfdea3b859c529777be3a0ede27c Mon Sep 17 00:00:00 2001
From: Delia Passalacqua <delia.passalacqua@garr.it>
Date: Mon, 9 May 2022 10:56:35 +0200
Subject: [PATCH] add istructions to use s3 endpoint with rclone

---
 .../kb/objstore/rclone_quick_tutorial.rst     | 98 ++++++++++++-------
 1 file changed, 63 insertions(+), 35 deletions(-)

diff --git a/web/support/kb/objstore/rclone_quick_tutorial.rst b/web/support/kb/objstore/rclone_quick_tutorial.rst
index 19c88c13..36104f59 100644
--- a/web/support/kb/objstore/rclone_quick_tutorial.rst
+++ b/web/support/kb/objstore/rclone_quick_tutorial.rst
@@ -23,7 +23,7 @@ Install it and then download rclone from:
 
 Unzip the archive and open the bash in the rclone directory. From here, you can use rclone command as follow::
 
-    ./rclone 
+    ./rclone
 
 
 Download OpenStack credentials
@@ -36,7 +36,7 @@ In order to use **Rclone**, you can either load the configuration from environme
 In both cases, you should edit the Rclone configuration file::
 
     $ nano .rclone.conf
-    
+
 Case 1: Take variables from environment
 """""""""""""""""""""""""""""""""""""""
 
@@ -49,7 +49,7 @@ Add the following text to *rclone.conf*::
 Then execute the content of the file::
 
     $ source app-credentials.sh
-   
+
 Case 2: Write variables in the configuration file
 """""""""""""""""""""""""""""""""""""""""""""""""
 Take note of these three variables in *app-credentials.sh*::
@@ -57,7 +57,7 @@ Take note of these three variables in *app-credentials.sh*::
     OS_REGION_NAME
     OS_APPLICATION_CREDENTIAL_ID
     OS_APPLICATION_CREDENTIAL_SECRET
-    
+
 Add the following text to *rclone.conf*::
 
     [garr-cloud]
@@ -69,7 +69,41 @@ Add the following text to *rclone.conf*::
     application_credential_secret = <insert here the content of OS_APPLICATION_CREDENTIAL_SECRET>
 
 Mind that ``env_auth = true`` takes variables from environment, so you shouldn't insert it in this case.
-    
+
+Case 3: Use EC2 credentials
+""""""""""""""""""""""""""
+First, you need to install the Openstack cli as described here in the `cli tutorial <https://cloud.garr.it/compute/install-cli/>`.
+
+Then execute the content of the file::
+
+    $ source app-credentials.sh
+
+And create the ec2 credentials::
+
+    $ openstack ec2 credentials create
+
+Take note of these two variables::
+
+    +------------+-----------------------+
+    | Field      | Value                 |
+    +------------+-----------------------+
+    | access     | <access_key>          |
+    | secret     | <secret_key>          |
+    +------------+-----------------------+
+
+Add the following text to *rclone.conf*::
+
+    [garr-cloud]
+    type = s3
+    provider = AWS
+    access_key_id =  <insert here the content of access_key>
+    secret_access_key =  <insert here the content of secret_key>
+    endpoint = https://swift.cloud.garr.it
+
+.. note::
+
+    You can use EC2 credentials to access object storage with other tools. Check `S3 interface to object storage <https://cloud.garr.it/support/kb/objstore/s3_quick_tutorial/>`.
+
 Check configuration
 ^^^^^^^^^^^^^^^^^^^
 .. note::
@@ -116,7 +150,7 @@ Now, suppose you have these files on your local filesystem::
         total 8
         -rw-rw-r-- 1 ubuntu ubuntu 1103 Nov 13 15:31 file1.txt
         drwxrwxr-x 2 ubuntu ubuntu 4096 Nov 13 15:32 subdir1
-        
+
         /tmp/test_dir/subdir1:
         total 4
         -rw-rw-r-- 1 ubuntu ubuntu 459 Nov 13 15:32 file2.txt
@@ -130,8 +164,8 @@ Execute the following command to synchronize it with the remote::
         459 sublevel/subdir1/file2.txt
 
 **Mind behaviour of sync!** It makes destination identical to source
-        
-        
+
+
 Copy files from remote to local
 -------------------------------
 
@@ -143,14 +177,14 @@ The following command copies files from remote to a local directory, create it i
         Checks:                 0 / 0, -
         Transferred:            2 / 2, 100%
         Elapsed time:       300ms
-        
+
     $ ls -lR checkDir/
         checkDir/:
         total 12
         -rw-rw-r-- 1 ubuntu ubuntu 1103 Nov 13 15:31 file1.txt
         -rw-rw-r-- 1 ubuntu ubuntu 1103 Nov 13 15:28 sample_file.txt
         drwxrwxr-x 2 ubuntu ubuntu 4096 Nov 13 15:53 subdir1
-        
+
         checkDir/subdir1:
         total 4
         -rw-rw-r-- 1 ubuntu ubuntu 459 Nov 13 15:32 file2.txt
@@ -161,9 +195,9 @@ Mounting object storage on local filesystem
 **Linux:**
 
 First, you need to create a directory on which you will mount your filesystem::
-    
+
     $ mkdir ~/mnt-rclone
-    
+
 Then you can simply mount your object storage with::
 
     $ rclone -vv --vfs-cache-mode writes mount garr-cloud: ~/mnt-rclone
@@ -172,15 +206,15 @@ Then you can simply mount your object storage with::
 
 First you have to download **Winsfp**:
 
-    http://www.secfs.net/winfsp/rel/ 
+    http://www.secfs.net/winfsp/rel/
 
 WinFsp is an open source Windows File System Proxy which provides a FUSE emulation layer.
 
 Then you can simply mount your object storage with (no need to create the directory in advance)::
 
-    ./rclone -vv --vfs-cache-mode writes mount garr-cloud: C:/mnt-rclone    
+    ./rclone -vv --vfs-cache-mode writes mount garr-cloud: C:/mnt-rclone
 
-``vfs-cache-mode`` flag enable file caching, you can use either  ``writes`` or ``full`` option. For further explanation you can see official documentation at the link: 
+``vfs-cache-mode`` flag enable file caching, you can use either  ``writes`` or ``full`` option. For further explanation you can see official documentation at the link:
 https://rclone.org/commands/rclone_mount/#file-caching
 
 Now that your object storage is mounted, you can list, create and delete files in it.
@@ -196,7 +230,7 @@ You can encrypt a directory in your remote container and decrypt it easily throu
 First, we create a new remote in the rclone configuration file. It will be a subdirectory of your working remote (i.e. ``garr-cloud`` remote and ``crypt-dir`` directory). So, everything inside ``garr-cloud:crypt-dir`` will be encrypted and anything outside won’t.::
 
     $ nano .rclone.conf
-    
+
 Copy and paste the following text at the end of the file::
 
     [garr-cloud-crypt]
@@ -204,21 +238,21 @@ Copy and paste the following text at the end of the file::
     remote = garr-cloud:crypt_dir
     filename_encryption = standard
     directory_name_encryption = true
-    
+
 Then set the passwords that will be saved obscured inside the config file::
-    
-    $ rclone config password garr-cloud-crypt password <type_a_password> 
-    $ rclone config password garr-cloud-crypt password2 <type_another_password> 
+
+    $ rclone config password garr-cloud-crypt password <type_a_password>
+    $ rclone config password garr-cloud-crypt password2 <type_another_password>
 
 Now we need to create ``crypt_dir`` directory inside ``garr-cloud``::
 
     $ rclone mkdir garr-dev:crypt_dir
-    
+
 or, if you have ``garr-cloud`` still mounted on ``~/mnt-rclone``::
-    
+
     $ mkdir ~/mnt-rclone/crypt_dir
 
-Every file you will create inside ``garr-cloud-crypt`` container will be encrypted. 
+Every file you will create inside ``garr-cloud-crypt`` container will be encrypted.
 You can try it following these steps. First, we create a file to copy::
 
     $ echo "Hello world" > test.txt
@@ -230,8 +264,8 @@ Then, we copy it to the remote, in a new directory named ``test_dir``::
 Now we can list the new created files through the mounted filesystem::
 
     $ cd ~/mnt-rclone/crypt_dir/
-    $ ls -R 
-    
+    $ ls -R
+
 You will get an output similar to this::
 
     .:
@@ -241,13 +275,13 @@ You will get an output similar to this::
     b4tc2rcdasquuns71k9fa2uiss
 
 and if you cat the content of the file, you will see that it has been encrypted::
-    
+
     $ cat 0ruoo4gjnnuk01p4gok56li8ts/b4tc2rcdasquuns71k9fa2uiss
-    
+
 To access and decrypt the file in a complete transparent way, you can copy it from the remote through::
 
     $ rclone ls garr-cloud-crypt:
-    
+
         1048576 test_dir/test.txt
 
     $ rclone copy garr-cloud-crypt: new_dir
@@ -259,11 +293,5 @@ To access and decrypt the file in a complete transparent way, you can copy it fr
         new_dir/test_dir:
         test.txt
 
-    $ cat new_dir/test_dir/test.txt 
+    $ cat new_dir/test_dir/test.txt
         Hello world
-
-
-
-
-
-
-- 
GitLab