Tuesday, January 17, 2017

How to build a POC label for IPv4 convert to IPv6 on VirtualBox for SwiftStack Environment and OpenVPN

This lab assume you have a IPv4 ( SwiftStack Controller ) and SAIO ( Swift All in One ) Nodes. For mixing OS, I have Ubuntu 1404, CentOS6 and CentOS7 to increase complexity of assumptions.

Before start you might need a IPv6 address set. 

The open ipv6 you can get is from ipv6 generator

fd52:ed8b:e916::/48
This is a randomly generated 48-bit unique local IPv6 prefix as defined by RFC 4193. It can be used for local IPv6 networking. Everytime the regenerate button is pressed or the page is reloaded a new random prefix will be generated.
64-bit subnetting
Unique local addresses have 48-bit prefixes, leaving 16 bits for local subnetting. Below see the addresses of the first and last subnets.
Prefix
fd52:ed8b:e916::/48
1st subnet
fd52:ed8b:e916::/64
last subnet
fd52:ed8b:e916:ffff::/64
IPv4 local address equivalent
The IPv6 unique local addresses are used similarly to the IPv4 local adresses e.g. 10.0.0.0/8. Unlike their IPv4 counterpart IPv6 unique local addresses have a 40-bit random part. Therefore if you connect 2 or more unique local networks, by VPN for example, it's very unlikely to ever have address collisions. [+]
Converting Swift Controller Environment from IPv4 to IPv6 
1. On all the nodes, add an IPv6 address to the NICs assigned to:
  • Outward-facing Interface
  • Cluster-facing Interface
  • Data replication Interface

Ubuntu14:04 Controller:
ss01:~$ cat /etc/network/interfaces.d/eth1.cfg
# The primary network interface
auto eth1
iface eth1 inet6 static
address fd52:ed8b:e916::0001
netmask 64
vagrant@ss01:~$ cat /etc/network/interfaces.d/eth1:0.cfg
# The primary network interface
auto eth1:0
iface eth1:0 inet static
address 172.28.128.23
netmask 255.255.255.0

CentOS7.2 SwiftNode 01:
ss02 $ cat /etc/sysconfig/network-scripts/ifcfg-enp0s8
#VAGRANT-BEGIN
# The contents below are automatically generated by Vagrant. Do not modify.
NM_CONTROLLED=no
BOOTPROTO=none
ONBOOT=yes
IPADDR=172.28.128.24
NETMASK=255.255.255.0
DEVICE=enp0s8
PEERDNS=no
#VAGRANT-END
IPV6INIT=yes
IPV6ADDR=fd52:ed8b:e916::0002

CentOS6.8 SwiftNode 02:
ss03 $ cat /etc/sysconfig/network-scripts/ifcfg-eth1
#VAGRANT-BEGIN
# The contents below are automatically generated by Vagrant. Do not modify.
NM_CONTROLLED=no
BOOTPROTO=none
ONBOOT=yes
IPADDR=172.28.128.25
NETMASK=255.255.255.0
IPV6INIT=yes
IPV6ADDR=fd52:ed8b:e916::0003
DEVICE=eth1
PEERDNS=no
#VAGRANT-END

$ ifconfig
$ ping6 fd52:ed8b:e916::0001
$ ping6 fd52:ed8b:e916::0002
$ ping6 fd52:ed8b:e916::0003

2. Restart network service on the nodes and confirm that the IPv6 addresses are working properly.
  1. Ubuntu
    1. sudo ifdown eth0 && sudo ifup eth0
    2. sudo service network-manager restart
  2. CentOS
    1. sudo service network restart
    2. sudo systemctl restart network.service

3. Reconfigure Interfaces in Swift Controller ( If you run SAIO you can ignore it but you need to manual push the Ring with new IPv6 IPs )
In the controller web UI, go to the node’s management page and 
  1. click on the “Network” button located at the left hand corner of the page.
  2. On the “Edit Network Interfaces” page,
  3. change the IP address assigned to those interfaces to the IPv6 addresses you setup in step (1).

4. Click the “Reassign Interface” button when done. It will prompt you to do a “config push” for the settings to take effect. However, you should not push a config until you have modified the network on all the nodes in your cluster.

5. Configure Cluster IP ( Ignore if you are run SAIO )
Go to the cluster management page and click on “Configure” in the left side menu. If your cluster has the “Cluster API Hostname” set, the only think you need to do is to ensure that you have an IPv6 record for the “Cluster API Hostname.” If, however, your cluster can only be reached using an IP address, you need to modify the “Cluster API IP Address” and change it from the IPv4 address to your new IPv6 address for the cluster.

6. Push the new config to the cluster and confirm that the config is pushed successfully. ( Ignore if you are not using Swift Controller )
(There is a chance where you will see
“SwiftStack Node Connectivity: NOT OK: IP service location(s): fd57:bd44:c845:e36a::3:58318 fd57:bd44:c845:e36a::4:58318 not reachable” warning. If that’s the case, please restart the ssnoded daemon on each node and it should resolve the condition).

$ sudo systemctl restart ssnoded <CentOS7>
$ sudo restart ssnoded <CentOS6>

7. To verify that everything worked, using a test account, try to authenticate to the cluster using its auth URL, make a request using the storage URL, and check that the Swift Web Console still works with your new all-IPv6 configured cluster.

8. SSH to Swift controller and once logged in, check on the ring’s builder file to make sure that the ring now contains the IPv6 address of the nodes.

9. Optional: On the Swift nodes, remove the IPv4 address(es) from the interface configuration and restart the network interface for the settings to take effect.

10. PS: if your lab is setting up in VirtualBox, you might need to configure IPv6 Address Sets ( e.g fd52:ed8b:e916::10 ) as below.





 Converting SS OpenVPN from IPv4 to IPv6

Controller ( Ubuntu ) : openvpn Server

OpenVPN
In case customers want to use IPv6 on the VPN tunnel (tun0 & tun1) in the controller, these are the steps needed:
  1. In /etc/openvpn/server.conf, add the following line to the configuration:
    server-ipv6 2001:db8:0:123::/64 (or you can change this to any other IPv6 address block)

  1. Add the following line at the end of /etc/openvpn/server.conf:
    #additional IPv6 setting
    tun-ipv6
    push tun-ipv6
    ifconfig-ipv6 2001:db8:0:123::1 2001:db8:0:123::2 (please pick 2 other IPv6 addresses if you are NOT using the block listed above. Note: 2001:db8:0:123::0 /64 is an IPv6 NAT address).

  1. Allow packet forwarding for IPv6 by uncommenting (or adding) “net.ipv6.conf.all.forwarding=1” line in /etc/sysctl.conf.
# Uncomment the next line to enable packet forwarding for IPv6
#  Enabling this option disables Stateless Address Autoconfiguration
#  based on Router Advertisements for this host
net.ipv6.conf.all.forwarding=1


  1. Restart OpenVPN services. ( Ubuntu Server )
$ service openvpn restart
 * Stopping virtual private network daemon(s)...                                                                                                         *   Stopping VPN 'recovery-server'                                                                                                                     rm: cannot remove ‘/run/openvpn/recovery-server.pid’: Permission denied
vagrant@ss01:~$ sudo service openvpn restart
 * Stopping virtual private network daemon(s)...                                                                                                         *   Stopping VPN 'recovery-server'                                                                                                              [ OK ]
 *   Stopping VPN 'server'                                                                                                                       [ OK ]
 * Starting virtual private network daemon(s)...                                                                                                         *   Autostarting VPN 'recovery-server'                                                                                                                  *   Autostarting VPN 'server'

Swift Node ( CentOS ) : openvpn client

Cent OS 7
Leverage systemctl -a find out openvpn client dameon

$ sudo systemctl -a | grep vpn
  openvpn@120e8807-d9c4-11e6-95af-0800270c4edc.service                                                           loaded    active   running   OpenVPN Robust And Highly Flexible Tunneling Application On 120e8807/d9c4/11e6/95af/0800270c4edc
  system-openvpn.slice                                                                                           loaded    active   active    system-openvpn.slice
$ sudo systemctl restart openvpn@120e8807-d9c4-11e6-95af-0800270c4edc.service

CentOS 6
$ service openvpn restart
Shutting down openvpn:                                     [  OK  ]
Starting openvpn:                                          [  OK  ]


Tuesday, January 3, 2017

How to setup GKE connection on your local

Before we start everything, you need to install gcloud at your local.
The way for installation, you can find in here.

$ gcloud init
Welcome! This command will take you through the configuration of gcloud.

Your current configuration has been set to: [default]

You can skip diagnostics next time by using the following flag:
  gcloud init --skip-diagnostics

Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
Reachability Check passed.
Network diagnostic (1/1 checks) passed.

You must log in to continue. Would you like to log in (Y/n)?  Y

Your browser has been opened to visit:



You are logged in as: [jwang@swiftstack.com].

Pick cloud project to use:
 [1] distributed-rendering
 [2] summer-optics-152017
Please enter numeric choice or text value (must exactly match list
item):  1

Your current project has been set to: [distributed-rendering].

Do you want to configure Google Compute Engine
(https://cloud.google.com/compute) settings (Y/n)?  Y

Which Google Compute Engine zone would you like to use as project
default?
If you do not specify a zone via a command line flag while working
with Compute Engine resources, the default is assumed.
 [1] asia-east1-b
 [2] asia-east1-a
 [3] asia-east1-c
 [4] asia-northeast1-c
 [5] asia-northeast1-b
 [6] asia-northeast1-a
 [7] europe-west1-d
 [8] europe-west1-b
 [9] europe-west1-c
 [10] us-central1-c
 [11] us-central1-f
 [12] us-central1-a
 [13] us-central1-b
 [14] us-east1-d
 [15] us-east1-c
 [16] us-east1-b
 [17] us-west1-a
 [18] us-west1-b
 [19] Do not set default zone
Please enter numeric choice or text value (must exactly match list
item):  10

Your project default Compute Engine zone has been set to [us-central1-c].
You can change it by running [gcloud config set compute/zone NAME].

Your project default Compute Engine region has been set to [us-central1].
You can change it by running [gcloud config set compute/region NAME].

Created a default .boto configuration file at [/Users/jwang/.boto]. See this file and
information about configuring Google Cloud Storage.
Your Google Cloud SDK is configured and ready to use!

* Commands that require authentication will use jwang@swiftstack.com by default
* Commands will reference project `distributed-rendering` by default
* Compute Engine commands will use region `us-central1` by default
* Compute Engine commands will use zone `us-central1-c` by default

Run `gcloud help config` to learn how to change individual settings

This gcloud configuration is called [default]. You can create additional configurations if you work with multiple accounts and/or projects.
Run `gcloud topic configurations` to learn more.

Some things to try next:

* Run `gcloud --help` to see the Cloud Platform services you can interact with. And run `gcloud help COMMAND` to get help on any gcloud command.
* Run `gcloud topic -h` to learn about advanced features of the SDK like arg files and output formatting


$ gcloud container clusters get-credentials ffmpeg-cluster \
>     --zone us-central1-c --project distributed-rendering
WARNING: Accessing a Container Engine cluster requires the kubernetes commandline
client [kubectl]. To install, run
  $ gcloud components install kubectl

Fetching cluster endpoint and auth data.
kubeconfig entry generated for ffmpeg-cluster.


$ gcloud components install kubectl


Your current Cloud SDK version is: 137.0.1
Installing components from version: 137.0.1

┌─────────────────────────────────────────────────┐
       These components will be installed.      
├────────────────────────────┬─────────┬──────────┤
            Name            │ Version │   Size  
├────────────────────────────┼─────────┼──────────┤
│ kubectl                                      
│ kubectl (Mac OS X, x86_64) │   1.4.6 │ 13.4 MiB │
└────────────────────────────┴─────────┴──────────┘

For the latest full release notes, please visit:

Do you want to continue (Y/n)?  Y

╔════════════════════════════════════════════════════════════╗
╠═ Creating update staging area                             ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Installing: kubectl                                      ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Installing: kubectl (Mac OS X, x86_64)                   ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Creating backup and activating new installation          ═╣
╚════════════════════════════════════════════════════════════╝

Performing post processing steps...done.

Update done!

Follow the set up from this link


1.  The environment variable GOOGLE_APPLICATION_CREDENTIALS is checked. If this variable is specified it should point to a file that defines the credentials. The simplest way to get a credential for this purpose is to create a Service account key in the Google API Console:
3.  From the project drop-down, select your project.
4.  On the Credentials page, select the Create credentials drop-down, then select Service account key.
5.  From the Service account drop-down, select an existing service account or create a new one.
6.  For Key type, select the JSON key option, then select Create. The file automatically downloads to your computer.
7.  Put the *.json file you just downloaded in a directory of your choosing. This directory must be private (you can't let anyone get access to this), but accessible to your web server code.
8.  Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the JSON file downloaded.
$ mv ./Downloads/Distributed\ Rendering-0818d5fb7809.json ./Documents/gce/
$ cd ./Documents/gce/
$ pwd
/Users/jwang/Documents/gce
$ export GOOGLE_APPLICATION_CREDENTIALS=/Users/jwang/Documents/gce/Distributed\ Rendering-0818d5fb7809.json
$ env | grep 'GOOGLE_APPLICATION_CREDENTIALS'
GOOGLE_APPLICATION_CREDENTIALS=/Users/jwang/Documents/gce/Distributed Rendering-0818d5fb7809.json
9.If you have installed the Google Cloud SDK on your machine and have run the command gcloud auth application-default login, your identity can be used as a proxy to test code calling APIs from that machine.


$ gcloud beta auth application-default login
You do not currently have this command group installed.  Using it
requires the installation of components: [beta]


Your current Cloud SDK version is: 137.0.1
Installing components from version: 137.0.1

┌─────────────────────────────────────────────┐
     These components will be installed.    
├──────────────────────┬────────────┬─────────┤
         Name           Version      Size 
├──────────────────────┼────────────┼─────────┤
│ gcloud Beta Commands │ 2016.01.12 │ < 1 MiB │
└──────────────────────┴────────────┴─────────┘

For the latest full release notes, please visit:

Do you want to continue (Y/n)?  Y

╔════════════════════════════════════════════════════════════╗
╠═ Creating update staging area                             ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Installing: gcloud Beta Commands                         ═╣
╠════════════════════════════════════════════════════════════╣
╠═ Creating backup and activating new installation          ═╣
╚════════════════════════════════════════════════════════════╝

Performing post processing steps...done.

Update done!

Restarting command:
  $ gcloud beta auth application-default login


The environment variable [GOOGLE_APPLICATION_CREDENTIALS] is set to:
  [/Users/jwang/Documents/gce/Distributed Rendering-0818d5fb7809.json]
Credentials will still be generated to the default location:
  [/Users/jwang/.config/gcloud/application_default_credentials.json]
To use these credentials, unset this environment variable before
running your application.

Do you want to continue (Y/n)?  Y

Your browser has been opened to visit:




Credentials saved to file: [/Users/jwang/.config/gcloud/application_default_credentials.json]

These credentials will be used by any library that requests
Application Default Credentials.

Now try to connect to your GKE cluster, there has "connect" button beside your cluster. Just click it.




$ gcloud container clusters get-credentials ffmpeg-cluster \
>     --zone us-central1-c --project distributed-rendering
Fetching cluster endpoint and auth data.
kubeconfig entry generated for ffmpeg-cluster.

$ kubectl proxy
Starting to serve on 127.0.0.1:8001