However, I was struggling for tempurl for pass days. Because during the days, I was dispirited and can't find too much logical info. People said about tempurl is more complicated than what's I thought and didn't explain the basic logic but just the cli (command lines).
Other than that, the some reference links includes the unit test cli but didn't give an overall summary. Here, I will explain the how curl and swift-client works to operate the tempurl and give a unit test example.
In sum, I really want give a quick intro about the basic logic, and after you aware of it, the cli for you will be logical as nature. Here are the items I would like to talk.
- tempurl basic logic with python example
- tempurl Lab Setup
- Server: Swift AIO setup
- Client: curl / swift-client setup
- tempurl configuration.
- tempurl unit test
- swift over swift vs swift over ceph.
- swift on ceph won't work regarding the limitation
1. tempurl basic logic with python example
For generate the tempurl from OpenStack Swift, the basic idea is you have a ID ( key ) with your account then you know where is your object your would like to share with other people. After that you hash the key and a expiration date/time with the link ( url ). That's it !- Key
- Path
- Expiration Date/Time
python example
The python example as below explain all the logics.
The info you need to assign.
- method: eg: GET
- host: if you test in your POC SAIO VM, you can give "http://127.0.0.1:8080"
- expiration: give the seconds for temp url expired.
- path: the path for your object, eg: '/v1/AUTH_test/testCon/test.txt'
- key: the key you would like to have or assign, eg: 'secret'
- sig: generate signature for combining 1, 2, 3 and 4
- tempurl: combine with hostname, object directory and temp_url_sign
--------python code start--------
'''
Created on Sep 21, 2015
@author: johnnywang
'''
import hmac
from hashlib import sha1
from time import time
#1
method = 'GET'
#2
host = "http://127.0.0.1:8080"
#3
duration_in_seconds = 6000 # Duration for which the url is valid
expires = int(time() + duration_in_seconds)
#4
path = '/v1/AUTH_test/testCon/test.txt'
#5
key = 'secret'
hmac_body = '%s\n%s\n%s' % (method, expires, path)
hmac_body = hmac.new(key, hmac_body, sha1).hexdigest()
#6
sig = hmac.new(key, hmac_body, sha1).hexdigest()
#7
rest_uri = "{host}{path}?temp_url_sig={sig}&temp_url_expires={expires}".format(
host=host, path=path, sig=sig, expires=expires)
print rest_uri
--------python code end--------
output:
http://127.0.0.1:8080/v1/AUTH_test/testCon/test.txt?temp_url_sig=efbd2cad15e098d2327b8c7109886882f4a7afec&temp_url_expires=1442870614
2. tempurl Lab setup ( Swift over Swift )
SAIO setup - Server
SAIO is swift all in one lab, you can follow the links as below I present before, setup a POC lab for verify tempurl quick. It's quick and easy.PS: you can configure port forwarding then you can ssh to your virtualbox SAIO vm.
There has two major client tools which can operate the swift, there are curl and swift-client.
curl setup - Client
in ubuntu
#sudo apt-get install curl libcurl3 libcurl3-devin mac
#brew install curlswift-client - Client
in ubuntu
#sudo aptitude install python-pip#sudo pip install python-swiftclient
in mac
#sudo easy_install pip#sudo pip install --upgrade setuptools
#sudo pip install python-swiftclient
http://thornelabs.net/2014/10/29/installing-python-swiftclient-on-os-x-yosemite.html
3. tempurl configuration
The key for make tempurl works is inject a key or second key in account level metadata. You can use curl or swift-client. Both should be all working well.Get token and endpoint url via Swift-client
- Get Auth-token or you username and password directly
export OS_STORAGE_URL=http://127.0.0.1:8080/v1/AUTH_swift
export OS_AUTH_TOKEN=AUTH_tk5050d2a92c43422e871d60cffa309022
swift@swift-VirtualBox:~$ swift -A http://127.0.0.1:8080/auth/v1.0 -U swift -K swift auth -v
export ST_AUTH=http://127.0.0.1:8080/auth/v1.0
export ST_USER=swift
export ST_KEY=swift
- Once you get the endpoint and token you can insert a key into account's metadata (X-Account-Meta-Temp-Url-Key: secret)
- Or you can access swift via username and password directly.
swift@swift-VirtualBox:~$ swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
Account: AUTH_test
Containers: 2
Objects: 0
Bytes: 0
Containers in policy "gold": 2
Objects in policy "gold": 0
Bytes in policy "gold": 0
Meta Temp-Url-Key: secret
X-Timestamp: 1439949170.11303
X-Trans-Id: tx81e1acbb06f544af90429-0055d5654d
Content-Type: text/plain; charset=utf-8
Accept-Ranges: bytes
Inject tempurl key
post "Temp-Url-Key"
swift@swift-VirtualBox:~$ swift --os-auth-token AUTH_tk29ec321ad87b43d0bfd8a7b687ab4a2f --os-storage-url http://127.0.0.1:8080/v1/AUTH_test post -m "Temp-URL-Key: secret"
Double check
using curl with token
swift@swift-VirtualBox:~$ curl -v -H 'X-Auth-Token: AUTH_tk71106acb07784da1859cd2e434eba109' http://127.0.0.1:8080/v1/AUTH_test/
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /v1/AUTH_test/ HTTP/1.1
> User-Agent: curl/7.35.0
> Host: 127.0.0.1:8080
> Accept: */*
> X-Auth-Token: AUTH_tk71106acb07784da1859cd2e434eba109
>
< HTTP/1.1 200 OK
< X-Account-Storage-Policy-Gold-Bytes-Used: 0
< Content-Length: 19
< X-Account-Storage-Policy-Gold-Object-Count: 0
< X-Account-Object-Count: 0
< X-Timestamp: 1439949170.11303
< X-Account-Meta-Temp-Url-Key: secret
< X-Account-Storage-Policy-Gold-Container-Count: 2
< X-Account-Bytes-Used: 0
< X-Account-Container-Count: 2
< Content-Type: text/plain; charset=utf-8
< Accept-Ranges: bytes
< X-Trans-Id: tx1d08eb3202cb44a49bc6e-0055d409fd
< Date: Wed, 19 Aug 2015 04:45:49 GMT
<
testCon
testFolder
* Connection #0 to host 127.0.0.1 left intact
swift@swift-VirtualBox:~$
via token
swift@swift-VirtualBox:~$ swift --os-auth-token AUTH_tk29ec321ad87b43d0bfd8a7b687ab4a2f --os-storage-url http://127.0.0.1:8080/v1/AUTH_test stat
Account: AUTH_test
Containers: 2
Objects: 0
Bytes: 0
Containers in policy "gold": 2
Objects in policy "gold": 0
Bytes in policy "gold": 0
Meta Temp-Url-Key: secret
X-Timestamp: 1439949170.11303
X-Trans-Id: tx2a025cd404df4b8c9c8c4-0055d55f12
Content-Type: text/plain; charset=utf-8
Accept-Ranges: bytes
via username/password
swift@swift-VirtualBox:~$ swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
Account: AUTH_test
Containers: 2
Objects: 0
Bytes: 0
Containers in policy "gold": 2
Objects in policy "gold": 0
Bytes in policy "gold": 0
Meta Temp-Url-Key: secret
X-Timestamp: 1439949170.11303
X-Trans-Id: tx81e1acbb06f544af90429-0055d5654d
Content-Type: text/plain; charset=utf-8
Accept-Ranges: bytes
As long as you can see the key in account level metadata, you can use the python script in above to put the path for the object and generate the tempurl.
PS: above is V1 authentication, here is V2 example if your keystone server is running V2 authentication.
swift@swift-VirtualBox:~$ swift -V 2.0 -A http://127.0.0.1:8080/auth/v2.0 -U swift -K swift post -m Temp-URL-Key:secret
swift@swift-VirtualBox:~$ swift -V 2.0 -A http://127.0.0.1:8080/auth/v2.0 -U swift -K swift stat -v
4. tempurl unit test
Here is the example you can use tempcurl to download the file, or you can copy the whole tempurl hyper-link and put on any browser to get the file.
swift@swift-VirtualBox:~$ curl -L -o 'download.txt' 'http://127.0.0.1:8080/v1/AUTH_test/testCon/test.txt?temp_url_sig=0003e838e9a253780e14b9a38545333c3255c119&temp_url_expires=1440014448'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 15 100 15 0 0 450 0 --:--:-- --:--:-- --:--:-- 483
swift@swift-VirtualBox:~$ cat download.txt
This is a TEST
PS: since download link with hash, you might want to use curl -L to trust the hash as a part of url, and -O is output into what's the name you would like to have for the object content.
5. tempurl for Swift over Ceph
Except the swift over swift, you can run swift over ceph which means leverage radosgateway for accessing your object in ceph. For POC, you can setup a Ceph lab follow these links.
- configure radosgateway: http://chianingwang.blogspot.com/2015/04/install-and-configure-ceph-radosgateway.html
tempurl available in RGW (http://ceph.com/docs/master/radosgw/swift/tempurl). unfortunately, it's not working in my lap or the environment I aware.
This is what’s I realized. For temp url, we need to inject a temp-url-key in account metadata, but in Swift over ceph (http://ceph.com/docs/master/radosgw/swift/) API, it doesn’t seem to allow us to change it but only read it. Without temp-url-key in account metadata, the tempurl will never work.
Feature
|
Status
|
Remarks
|
Authentication
|
Supported
|
|
Get Account Metadata
|
Supported
|
No custom metadata
|
Swift ACLs
|
Supported
|
Supports a subset of Swift ACLs
|
List Containers
|
Supported
|
|
Delete Container
|
Supported
|
|
Create Container
|
Supported
|
|
Get Container Metadata
|
Supported
|
|
Update Container Metadata
|
Supported
|
|
Delete Container Metadata
|
Supported
|
|
List Objects
|
Supported
|
|
Static Website
|
Not Supported
|
|
Create Object
|
Supported
|
|
Create Large Object
|
Supported
|
|
Delete Object
|
Supported
|
|
Get Object
|
Supported
|
|
Copy Object
|
Supported
|
|
Get Object Metadata
|
Supported
|
|
Update Object Metadata
|
Supported
|
|
Expiring Objects
|
Not Supported
|
|
Object Versioning
|
Not Supported
|
|
CORS
|
Not Supported
|
So if you know above statement is wrong or you figure the way out, please share with me.
Reference:
http://thornelabs.net/2014/10/29/installing-python-swiftclient-on-os-x-yosemite.html
https://swiftstack.com/docs/integration/python-swiftclient.html
https://swiftstack.com/docs/integration/python-swiftclient.html
