IT related stuff

Latest

Wish List, Common Misunderstanding, Undocumented – OpenStack Identity API – Authentication, Add User

I have came across the following issues during my intern work which is all about OpenStack API.

1 Wish List

1.1 Port Number

I find it inconvenient that port number is not documented within the API reference. Although the full urls are provided for some API endpoints in the “API access” page in the web portal ,  a lot other useful ports are not , it’s usage info scattered all over the place.  Here is a good answer provided by sngirame.

Nova-api
8773 (for EC2 API)
8774 (for openstack API)
8775 (metadata port)
3333 (when accessing S3 API)

nova-novncproxy
6080
5800/5900 (VNC)

cinder 8776

glance
9191 (glance registry)
9292 (glance api)

keystone
5000 (public port)
35357 (admin port)

http 80
Mysql 3306
AMQP 5672

1.2 Naming Conventions

It’s easy when only one person writes the code, but in a large project like OpenStack, it’s not that easy to keep everyone on the same page.  I wish we have the same name refers to the same parameters throughout the API, command line client and XML/WADL markup. Surprisingly obscure parameters/extensions often have much uniform names as people can not easily come up with an alternative. On the other hand the most commonly understandable parameters can have all sorts variations.

For example, username and name are both used, sometimes username in method request and name in response, sometimes the other way around, but command line tool only use option “–name” and it seems in the database it is only “name”, finally in the “open_rc.sh” configuration file generated it’s “OS_USERNAME”.

Another example would be Tenant ID, two words allows more creativity, so we have “tenantId”, “tenant_id”, “tenant” and “id”. Not to mention the confusion added by the concept of tenant itself.

I tend to go with the definition that tenant is “the entity that owns the resources”, so it can be a user or a project. Now I should stop before my head explode while trying to explain the difference between tenant, user, project, TENANT_NAME, USERNAME… Well, by the time I have to deal with then, I already know enough, so you always guess right. I feel it’s only confusing when you know too little or too much. When you pick up more and more similar terms that’s when you start to get confused again “Oh, what’s the name of the filed I should use this time?”

1.3 Keystone Command Line Client

It seems that keystone allows creation of a user without specifying password, and it doesn’t look like it has generated a temp password (since the RESTfull API is not returning any generated password, and there is no way to obtain the password, you can only reset it)? So I failed to see what’s the point of NOT making password a compulsory field? (Apart from providing another chance for users to make a mistake…) Although you can use the keystone user-password-update command to update the password later, but I failed to come up with a scenario that doing it in two steps being necessary? It feels more normal/common to create a user always with a password. See 3.2 for more details.

2 Common Misunderstanding

2.1 Authentication: Scoped Token

Most of the API functionality require authentication via the “X-Auth-Token” header, to obtain this token (using API 2.0), we need to POST to the host:port/2.0/tokens method.

Example of obtaining token from Keystone can be found here, this post also explains scoped vs unscoped token.

3 Everyone needs it probably knows it but not officially documented

3.1 Command Line Clients –debug Option

I found the --debug option with the command line clients can provide very useful info about how to use API.

ubuntu@openstack:~$ source ~/admin-openrc.sh
ubuntu@openstack:~$ keystone tenant-list
+----------------------------------+--------------------+---------+
|                id                |        name        | enabled |
+----------------------------------+--------------------+---------+
| 5652848cd478403ca195bd3c42e307c6 |       admin        |   True  |
| 371420f011eb42e4bd98121877e73afb |      alt_demo      |   True  |
| af615b6af60b497e860392ad5e9f8dab |        demo        |   True  |
| 1e301d5718864c8b80d47f42e73b7851 | invisible_to_admin |   True  |
| 85d205e5bc284ab99a430ef46040b468 |      service       |   True  |
+----------------------------------+--------------------+---------+

keystone --debug user-role-list
DEBUG:keystoneclient.session:REQ: curl -i -X POST http://13.102.155.7:5000/v2.0/tokens -H "Content-Type: application/json" -H "User-Agent: python-keystoneclient"
DEBUG:keystoneclient.session:REQ BODY: {"auth": {"passwordCredentials": {"username": "admin", "password": "password"}, "tenantId": "5652848cd478403ca195bd3c42e307c6"}}
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): 130.102.155.7
DEBUG:requests.packages.urllib3.connectionpool:"POST /v2.0/tokens HTTP/1.1" 200 7624
DEBUG:keystoneclient.session:RESP: [200] CaseInsensitiveDict({'date': 'Thu, 06 Feb 2014 03:02:35 GMT', 'vary': 'X-Auth-Token', 'content-length': '7624', 'content-type': 'application/json'})
RESP BODY: {"access": {"token": {"issued_at": "2014-02-06T03:02:35.802990", "expires": "2014-02-06T04:02:35Z", "id": "xx=", "tenant": {"description": null, "enabled": true, "id": "5652848cd478403ca195bd3c42e307c6", "name": "admin"}}, "serviceCatalog": [{"endpoints": [{"adminURL": "http://130.102.155.7:8774/v2/5652848cd478403ca195bd3c42e307c6", "region": "RegionOne", "internalURL": "http://130.102.155.7:8774/v2/5652848cd478403ca195bd3c42e307c6", "id": "92976d9808574401b45494789ebf7ece", "publicURL": "http://130.102.155.7:8774/v2/5652848cd478403ca195bd3c42e307c6"}], "endpoints_links": [], "type": "compute", "name": "nova"}, {"endpoints": [{"adminURL": "http://130.102.155.7:8776/v2/5652848cd478403ca195bd3c42e307c6", "region": "RegionOne", "internalURL": "http://130.102.155.7:8776/v2/5652848cd478403ca195bd3c42e307c6", "id": "8f11b29083534ac4abec78c1a40b733d", "publicURL": "http://130.102.155.7:8776/v2/5652848cd478403ca195bd3c42e307c6"}], "endpoints_links": [], "type": "volumev2", "name": "cinderv2"}, {"endpoints": [{"adminURL": "http://130.102.155.7:8774/v3", "region": "RegionOne", "internalURL": "http://130.102.155.7:8774/v3", "id": "4d972889effe4ecaa25ee9579b2b72d9", "publicURL": "http://130.102.155.7:8774/v3"}], "endpoints_links": [], "type": "computev3", "name": "novav3"}, {"endpoints": [{"adminURL": "http://130.102.155.7:3333", "region": "RegionOne", "internalURL": "http://130.102.155.7:3333", "id": "33375cdedf7a4e7da68b89adddb49232", "publicURL": "http://130.102.155.7:3333"}], "endpoints_links": [], "type": "s3", "name": "s3"}, {"endpoints": [{"adminURL": "http://130.102.155.7:9292", "region": "RegionOne", "internalURL": "http://130.102.155.7:9292", "id": "47376f3d62384662a087d029e84df97c", "publicURL": "http://130.102.155.7:9292"}], "endpoints_links": [], "type": "image", "name": "glance"}, {"endpoints": [{"adminURL": "http://130.102.155.7:8776/v1/5652848cd478403ca195bd3c42e307c6", "region": "RegionOne", "internalURL": "http://130.102.155.7:8776/v1/5652848cd478403ca195bd3c42e307c6", "id": "1b9a3c1abdeb4097a2f363f5ad49e7eb", "publicURL": "http://130.102.155.7:8776/v1/5652848cd478403ca195bd3c42e307c6"}], "endpoints_links": [], "type": "volume", "name": "cinder"}, {"endpoints": [{"adminURL": "http://130.102.155.7:8773/services/Admin", "region": "RegionOne", "internalURL": "http://130.102.155.7:8773/services/Cloud", "id": "0fdf95fa85b446a9aba4f855b8c5bcd7", "publicURL": "http://130.102.155.7:8773/services/Cloud"}], "endpoints_links": [], "type": "ec2", "name": "ec2"}, {"endpoints": [{"adminURL": "http://130.102.155.7:35357/v2.0", "region": "RegionOne", "internalURL": "http://130.102.155.7:5000/v2.0", "id": "2d05144069dd4a60856eddb229e0c17d", "publicURL": "http://130.102.155.7:5000/v2.0"}], "endpoints_links": [], "type": "identity", "name": "keystone"}], "user": {"username": "admin", "roles_links": [], "id": "ad77accec30e49e6a8190f44ff38a312", "roles": [{"name": "admin"}], "name": "admin"}, "metadata": {"is_admin": 0, "roles": ["3cc3fcee03f9494bbeeefdccffd5e4b2"]}}}

DEBUG:iso8601.iso8601:Parsed 2014-02-06T04:02:35Z into {'tz_sign': None, 'second_fraction': None, 'hour': u'04', 'tz_hour': None, 'month': u'02', 'timezone': u'Z', 'second': u'35', 'tz_minute': None, 'year': u'2014', 'separator': u'T', 'day': u'06', 'minute': u'02'} with default timezone
DEBUG:iso8601.iso8601:Got u'2014' for 'year' with default None
DEBUG:iso8601.iso8601:Got u'02' for 'month' with default None
DEBUG:iso8601.iso8601:Got u'06' for 'day' with default None
DEBUG:iso8601.iso8601:Got u'04' for 'hour' with default None
DEBUG:iso8601.iso8601:Got u'02' for 'minute' with default None
DEBUG:iso8601.iso8601:Got u'35' for 'second' with default None
DEBUG:iso8601.iso8601:Parsed 2014-02-06T04:02:35Z into {'tz_sign': None, 'second_fraction': None, 'hour': u'04', 'tz_hour': None, 'month': u'02', 'timezone': u'Z', 'second': u'35', 'tz_minute': None, 'year': u'2014', 'separator': u'T', 'day': u'06', 'minute': u'02'} with default timezone
DEBUG:iso8601.iso8601:Got u'2014' for 'year' with default None
DEBUG:iso8601.iso8601:Got u'02' for 'month' with default None
DEBUG:iso8601.iso8601:Got u'06' for 'day' with default None
DEBUG:iso8601.iso8601:Got u'04' for 'hour' with default None
DEBUG:iso8601.iso8601:Got u'02' for 'minute' with default None
DEBUG:iso8601.iso8601:Got u'35' for 'second' with default None
DEBUG:keystoneclient.session:REQ: curl -i -X GET http://13.102.155.7:35357/v2.0/tenants/5652848cd478403ca195bd3c42e307c6/users/ad77accec30e49e6a8190f44ff38a312/roles -H "User-Agent: python-keystoneclient" -H "X-Auth-Token: MIINhAYJKoZIhvcNAQcCoIINdTCCDXECAQExCTAHBgUrDgMCGjCCC9oGCSqGSIb3DQEHAaCCC8sEggvHeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxNC0wMi0wNlQwMzowMjozNS44MDI5OTAiLCAiZXhwaXJlcyI6ICIyMDE0LTAyLTA2VDA0OjAyOjM1WiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogbnVsbCwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNTY1Mjg0OGNkNDc4NDAzY2ExOTViZDNjNDJlMzA3YzYiLCAibmFtZSI6ICJhZG1pbiJ9fSwgInNlcnZpY2VDYXRhbG9nIjogW3siZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6ODc3NC92Mi81NjUyODQ4Y2Q0Nzg0MDNjYTE5NWJkM2M0MmUzMDdjNiIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzQvdjIvNTY1Mjg0OGNkNDc4NDAzY2ExOTViZDNjNDJlMzA3YzYiLCAiaWQiOiAiOTI5NzZkOTgwODU3NDQwMWI0NTQ5NDc4OWViZjdlY2UiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzQvdjIvNTY1Mjg0OGNkNDc4NDAzY2ExOTViZDNjNDJlMzA3YzYifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiY29tcHV0ZSIsICJuYW1lIjogIm5vdmEifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTMwLjEwMi4xNTUuNzo4Nzc2L3YyLzU2NTI4NDhjZDQ3ODQwM2NhMTk1YmQzYzQyZTMwN2M2IiwgInJlZ2lvbiI6ICJSZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6ODc3Ni92Mi81NjUyODQ4Y2Q0Nzg0MDNjYTE5NWJkM2M0MmUzMDdjNiIsICJpZCI6ICI4ZjExYjI5MDgzNTM0YWM0YWJlYzc4YzFhNDBiNzMzZCIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6ODc3Ni92Mi81NjUyODQ4Y2Q0Nzg0MDNjYTE5NWJkM2M0MmUzMDdjNiJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJ2b2x1bWV2MiIsICJuYW1lIjogImNpbmRlcnYyIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6ODc3NC92MyIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzQvdjMiLCAiaWQiOiAiNGQ5NzI4ODllZmZlNGVjYWEyNWVlOTU3OWIyYjcyZDkiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzQvdjMifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiY29tcHV0ZXYzIiwgIm5hbWUiOiAibm92YXYzIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6MzMzMyIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43OjMzMzMiLCAiaWQiOiAiMzMzNzVjZGVkZjdhNGU3ZGE2OGI4OWFkZGRiNDkyMzIiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43OjMzMzMifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiczMiLCAibmFtZSI6ICJzMyJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43OjkyOTIiLCAicmVnaW9uIjogIlJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTMwLjEwMi4xNTUuNzo5MjkyIiwgImlkIjogIjQ3Mzc2ZjNkNjIzODQ2NjJhMDg3ZDAyOWU4NGRmOTdjIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTMwLjEwMi4xNTUuNzo5MjkyIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImltYWdlIiwgIm5hbWUiOiAiZ2xhbmNlIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6ODc3Ni92MS81NjUyODQ4Y2Q0Nzg0MDNjYTE5NWJkM2M0MmUzMDdjNiIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzYvdjEvNTY1Mjg0OGNkNDc4NDAzY2ExOTViZDNjNDJlMzA3YzYiLCAiaWQiOiAiMWI5YTNjMWFiZGViNDA5N2EyZjM2M2Y1YWQ0OWU3ZWIiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzYvdjEvNTY1Mjg0OGNkNDc4NDAzY2ExOTViZDNjNDJlMzA3YzYifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAidm9sdW1lIiwgIm5hbWUiOiAiY2luZGVyIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6ODc3My9zZXJ2aWNlcy9BZG1pbiIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzMvc2VydmljZXMvQ2xvdWQiLCAiaWQiOiAiMGZkZjk1ZmE4NWI0NDZhOWFiYTRmODU1YjhjNWJjZDciLCAicHVibGljVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzMvc2VydmljZXMvQ2xvdWQifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiZWMyIiwgIm5hbWUiOiAiZWMyIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6MzUzNTcvdjIuMCIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43OjUwMDAvdjIuMCIsICJpZCI6ICIyZDA1MTQ0MDY5ZGQ0YTYwODU2ZWRkYjIyOWUwYzE3ZCIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6NTAwMC92Mi4wIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImlkZW50aXR5IiwgIm5hbWUiOiAia2V5c3RvbmUifV0sICJ1c2VyIjogeyJ1c2VybmFtZSI6ICJhZG1pbiIsICJyb2xlc19saW5rcyI6IFtdLCAiaWQiOiAiYWQ3N2FjY2VjMzBlNDllNmE4MTkwZjQ0ZmYzOGEzMTIiLCAicm9sZXMiOiBbeyJuYW1lIjogImFkbWluIn1dLCAibmFtZSI6ICJhZG1pbiJ9LCAibWV0YWRhdGEiOiB7ImlzX2FkbWluIjogMCwgInJvbGVzIjogWyIzY2MzZmNlZTAzZjk0OTRiYmVlZWZkY2NmZmQ1ZTRiMiJdfX19MYIBgTCCAX0CAQEwXDBXMQswCQYDVQQGEwJVUzEOMAwGA1UECAwFVW5zZXQxDjAMBgNVBAcMBVVuc2V0MQ4wDAYDVQQKDAVVbnNldDEYMBYGA1UEAwwPd3d3LmV4YW1wbGUuY29tAgEBMAcGBSsOAwIaMA0GCSqGSIb3DQEBAQUABIIBAIago3RroyyNxxnutMyU8NttsUjws2IRHMVOch2B3sjGUTJfI4DWx+0CDpiH7iOVSRCeUQ0Tn0A62475Qf5pdLh9EYs+gKUD8WG7eqAXb6w3Ns6LaivvMIn6YO5-HJwhJYUYKrm-gTu+inlYcBEr0Hrk--RVn+eDGmaOfU-avLqc9r6yPnWfbX+qNMR9xI5F17fhB+Hz+J76EDGuGoZj7uPDiBNY+WA1z4kXdKuO8eUkNj89vaK76IVBgdgFrV9MQrLZDAGTJEh4trrRBRAZAQvbbWX5mGgoqL6r+xyJHP3AE2cwZjCStkRjbW5qC2y8dmkiOlKuK5VV6GRn8qnoVVQ="
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): 130.102.155.7
DEBUG:requests.packages.urllib3.connectionpool:"GET /v2.0/tenants/5652848cd478403ca195bd3c42e307c6/users/ad77accec30e49e6a8190f44ff38a312/roles HTTP/1.1" 200 72
DEBUG:keystoneclient.session:RESP: [200] CaseInsensitiveDict({'date': 'Thu, 06 Feb 2014 03:02:36 GMT', 'vary': 'X-Auth-Token', 'content-length': '72', 'content-type': 'application/json'})
RESP BODY: {"roles": [{"id": "3cc3fcee03f9494bbeeefdccffd5e4b2", "name": "admin"}]}

+----------------------------------+-------+----------------------------------+----------------------------------+
|                id                |  name |             user_id              |            tenant_id             |
+----------------------------------+-------+----------------------------------+----------------------------------+
| 3cc3fcee03f9494bbeeefdccffd5e4b2 | admin | ad77accec30e49e6a8190f44ff38a312 | 5652848cd478403ca195bd3c42e307c6 |
+----------------------------------+-------+----------------------------------+----------------------------------+

Since every token expires quickly (valid for 1 hour), the client request a new auth token every time making a new request.

3.2 Add User: POST v2.0/users

The supported parameters for this method are listed below, the one I’d like to emphasis on is the $customized_field.

Attribute Type Description
name string Required. The user name (must be unique).
password string Optional. Both “password” and “OS-KSADM:password” will work.
enabled boolean Optional. Initial user enabled status (default true).
tenantId string Optional. If this is not specified for a new user, attempting to login to the Cloud Control Panel will return error “You are not authorized for any projects”.
email string Optional. New user email address.
$customized_field string/boolean/number Optional. Can be anything, like the email field.

The following example shows an “add user” request with customized fields.

ubuntu@openstack:~$ curl -X POST -H "X-Auth-Token: xx=" -d '{"user":{"name":"name_6","pass-none-exist":"test-none-exist","non-existing":"non-existing", "anything-bool":false}}' -H 'Content-type: application/json' http://13.102.155.7:35357/v2.0/users

{"user": {"name": "name_6", "pass_none_exist": "test-none-exist", "enabled": true, "non_existing": "non-existing", "anything_bool": false, "id": "2fb0955de04b481ab39dd393837cd8b4"}}

ubuntu@openstack:~$ keystone user-get name_6

+-----------------+----------------------------------+
|     Property    |              Value               |
+-----------------+----------------------------------+
|  anything_bool  |              False               |
|     enabled     |               True               |
|        id       | 2fb0955de04b481ab39dd393837cd8b4 |
|       name      |              name_6              |
|   non_existing  |           non-existing           |
| pass_none_exist |         test-none-exist          |
+-----------------+----------------------------------+

4 MythBusters (What’s this ? Click here)

4.1 Obtain auth token with token?

Since some documentation claim there is option other than “passwordCredentials” which allow user to obtain auth token, I also did some further investigation which seems to show that “passwordCredentials” to be the only method.

{"error": {"message": "Expecting to find auth in request body. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.", "code": 400, "title": "Bad Request"}}
{"error": {"message": "Expecting to find passwordCredentials in auth. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.", "code": 400, "title": "Bad Request"}}

As the above errors indicate that the expected structure of the request clearly follows the structure (with passwordCredentials being compulsory) :

{
   "auth":{
      "passwordCredentials":{
         "username":"something",
         "password":"some_password"
      }
   }
}

4.2 Admin can only create a max of 100 users?

Some suggested that there may be a max limit of 100 users an admin can create so I tested with the following script.

#!/bin/bash
for i in $(seq 0 100);
do
	json="{\"user\":{\"name\":\"name_$i\"}}"
	curl -X POST -H "X-Auth-Token: MIINhAYJKoZIhvcNAQcCoIINdTCCDXECAQExCTAHBgUrDgMCGjCCC9oGCSqGSIb3DQEHAaCCC8sEggvHeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxNC0wMi0wNlQwODoxNToyNi42ODczNjQiLCAiZXhwaXJlcyI6ICIyMDE0LTAyLTA2VDA5OjE1OjI2WiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogbnVsbCwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiNTY1Mjg0OGNkNDc4NDAzY2ExOTViZDNjNDJlMzA3YzYiLCAibmFtZSI6ICJhZG1pbiJ9fSwgInNlcnZpY2VDYXRhbG9nIjogW3siZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6ODc3NC92Mi81NjUyODQ4Y2Q0Nzg0MDNjYTE5NWJkM2M0MmUzMDdjNiIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzQvdjIvNTY1Mjg0OGNkNDc4NDAzY2ExOTViZDNjNDJlMzA3YzYiLCAiaWQiOiAiOTI5NzZkOTgwODU3NDQwMWI0NTQ5NDc4OWViZjdlY2UiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzQvdjIvNTY1Mjg0OGNkNDc4NDAzY2ExOTViZDNjNDJlMzA3YzYifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiY29tcHV0ZSIsICJuYW1lIjogIm5vdmEifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTMwLjEwMi4xNTUuNzo4Nzc2L3YyLzU2NTI4NDhjZDQ3ODQwM2NhMTk1YmQzYzQyZTMwN2M2IiwgInJlZ2lvbiI6ICJSZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6ODc3Ni92Mi81NjUyODQ4Y2Q0Nzg0MDNjYTE5NWJkM2M0MmUzMDdjNiIsICJpZCI6ICI4ZjExYjI5MDgzNTM0YWM0YWJlYzc4YzFhNDBiNzMzZCIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6ODc3Ni92Mi81NjUyODQ4Y2Q0Nzg0MDNjYTE5NWJkM2M0MmUzMDdjNiJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJ2b2x1bWV2MiIsICJuYW1lIjogImNpbmRlcnYyIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6ODc3NC92MyIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzQvdjMiLCAiaWQiOiAiNGQ5NzI4ODllZmZlNGVjYWEyNWVlOTU3OWIyYjcyZDkiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzQvdjMifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiY29tcHV0ZXYzIiwgIm5hbWUiOiAibm92YXYzIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6MzMzMyIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43OjMzMzMiLCAiaWQiOiAiMzMzNzVjZGVkZjdhNGU3ZGE2OGI4OWFkZGRiNDkyMzIiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43OjMzMzMifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiczMiLCAibmFtZSI6ICJzMyJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43OjkyOTIiLCAicmVnaW9uIjogIlJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTMwLjEwMi4xNTUuNzo5MjkyIiwgImlkIjogIjQ3Mzc2ZjNkNjIzODQ2NjJhMDg3ZDAyOWU4NGRmOTdjIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTMwLjEwMi4xNTUuNzo5MjkyIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImltYWdlIiwgIm5hbWUiOiAiZ2xhbmNlIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6ODc3Ni92MS81NjUyODQ4Y2Q0Nzg0MDNjYTE5NWJkM2M0MmUzMDdjNiIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzYvdjEvNTY1Mjg0OGNkNDc4NDAzY2ExOTViZDNjNDJlMzA3YzYiLCAiaWQiOiAiMWI5YTNjMWFiZGViNDA5N2EyZjM2M2Y1YWQ0OWU3ZWIiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzYvdjEvNTY1Mjg0OGNkNDc4NDAzY2ExOTViZDNjNDJlMzA3YzYifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAidm9sdW1lIiwgIm5hbWUiOiAiY2luZGVyIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6ODc3My9zZXJ2aWNlcy9BZG1pbiIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzMvc2VydmljZXMvQ2xvdWQiLCAiaWQiOiAiMGZkZjk1ZmE4NWI0NDZhOWFiYTRmODU1YjhjNWJjZDciLCAicHVibGljVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43Ojg3NzMvc2VydmljZXMvQ2xvdWQifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiZWMyIiwgIm5hbWUiOiAiZWMyIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6MzUzNTcvdjIuMCIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMzAuMTAyLjE1NS43OjUwMDAvdjIuMCIsICJpZCI6ICIyZDA1MTQ0MDY5ZGQ0YTYwODU2ZWRkYjIyOWUwYzE3ZCIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEzMC4xMDIuMTU1Ljc6NTAwMC92Mi4wIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImlkZW50aXR5IiwgIm5hbWUiOiAia2V5c3RvbmUifV0sICJ1c2VyIjogeyJ1c2VybmFtZSI6ICJhZG1pbiIsICJyb2xlc19saW5rcyI6IFtdLCAiaWQiOiAiYWQ3N2FjY2VjMzBlNDllNmE4MTkwZjQ0ZmYzOGEzMTIiLCAicm9sZXMiOiBbeyJuYW1lIjogImFkbWluIn1dLCAibmFtZSI6ICJhZG1pbiJ9LCAibWV0YWRhdGEiOiB7ImlzX2FkbWluIjogMCwgInJvbGVzIjogWyIzY2MzZmNlZTAzZjk0OTRiYmVlZWZkY2NmZmQ1ZTRiMiJdfX19MYIBgTCCAX0CAQEwXDBXMQswCQYDVQQGEwJVUzEOMAwGA1UECAwFVW5zZXQxDjAMBgNVBAcMBVVuc2V0MQ4wDAYDVQQKDAVVbnNldDEYMBYGA1UEAwwPd3d3LmV4YW1wbGUuY29tAgEBMAcGBSsOAwIaMA0GCSqGSIb3DQEBAQUABIIBAA649hWeXRkii0Nqo3qJ45OON27wAxiQPurXTY1h5j3cGniJkZfCoez1QswIkMJgmcN4Xk+KP5Jl97wYfEiaRRIQ+3udFbU83CKfUEiS63HxOSquqx+2TwShZTeyxHDAPiLxNvWhL9oAOiV2bFoMxhUN25DhnMvfxWFB7xST+2tT9vCKF-oBrL7iqBKrjwunXHwt+htmUkWKWNt5VHGORkRwObAAfYlY-uPl-DoJOv-8YzT4yQNyO-jDxM5MvYTdnb3pUIRNKdLNaKsFgj6ygcD9stCiRW4IIr-I9+tGUofepJqt5LExaCS4sJbn7VqA1jcQt9CmxvMye5GnGxh6o2I=" -d $json -H 'Content-type: application/json' http://130.102.155.7:35357/v2.0/users
done

I did NOT hit any limit with 100 users, neither can I say if there is a limit, so I’ll leave it here for now.

Testing Environment & Command-line Utility & Editor – OPW Experience with OpenStack Part 2

This is a follow up of the last post with more things I learned.

Testing Environment

It turned out the best option to testing out the OpenStack API is to install my own using DevStack. It’s understandable that cloud providers do not want to give general user too much power.

I end up install DevStack on a small Nectar instance, with the following specs:

Flavor:m1.small
RAM:4GB
VCPUs:1 VCPU
Disk:10GB

Everything went smoothly, although it says

stack.sh completed in 5035 seconds

I felt it took more than 2 hours.

Command-line Utility

Command-line tools can often save some time/effort doing repetitive tasks.

I was given a task which involves:

Identify which code samples here:

http://git.openstack.org/cgit/openstack/api-site/tree/api-ref/src/wadls/compute-api/src/api_samples

Do not have a corresponding WADL file here:

http://git.openstack.org/cgit/openstack/api-site/tree/api-ref/src/wadls/compute-api/src/ext

It doesn’t have to be done using script, “Ctrl + F” search in the browser can do the job, but I don’t like the sound of “manually” search the name of the file every time. So I prepared the following bash script, it doesn’t really save me much time (compare to the alternative), but certainly provides a reusable solution.

#!/bin/bash
BASE_PATH="/home/ubuntu/api-site/api-ref/src/wadls/compute-api/"
SAMPLE_PATH="/home/ubuntu/api-site/api-ref/src/wadls/compute-api/src/api_samples"
WADL_PATH="/home/ubuntu/api-site/api-ref/src/wadls/compute-api/src/ext"
SAMPLE_FILE="/tmp/sample.tmp"
WADL_FILE="/tmp/wadl.tmp"
TEMP_FILE="/tmp/missing_wadl.tmp"
BRANCH="latest"
JSON=".json"
XML=".xml"
WADL=".wadl"
cd $BASE_PATH
git checkout $BRANCH
git pull
ls $SAMPLE_PATH > $SAMPLE_FILE
ls $WADL_PATH > $WADL_FILE

# option -i means in place
sed -i "s/$JSON//g" "$SAMPLE_FILE"
sed -i "s/$XML//g" "$SAMPLE_FILE"
sed -i "s/$WADL//g" "$WADL_FILE"
uniq < $SAMPLE_FILE > $TEMP_FILE && mv $TEMP_FILE $SAMPLE_FILE

#select Case Insensitive, non matching lines from $SAMPLE_FILE
grep -i -v -f $WADL_FILE $SAMPLE_FILE
rm $WADL_FILE
rm $SAMPLE_FILE

It can be run as:

ubuntu@openstack:~$ sh missing_wadl.sh

Output:

Already on 'latest'
Already up-to-date.
addresses
address
all_extensions
extension
flavor-get-resp
flavors-list-resp
image-get-resp
image-metadata-get-resp
image-metadata-post-req
image-metadata-post-resp
image-metadata-put-req
image-metadata-put-resp
image-metadata-resp
image-meta-key-get
image-meta-key-put-req
image-meta-key-put-resp
images-details-get-resp
images-details-resp
images-list-get-resp
images-list-resp
limit-get-resp
NMN
os-assisted-volume-snapshots
os-availability-zone
os-baremetal-nodes
os-cell-capacities
os-cells
os-config-drive
OS-DCF
os-evacuate
os-extended-floating-ips
os-extended-quotas
os-extended-volumes
OS-EXT-SRV-ATTR
OS-EXT-STS
OS-EXT-VIF-NET
os-flavor-manage
os-flavor-swap
OS-FLV-EXT-DATA
os-fping
os-hide-server-addresses
os-multiple-create
os-quota-class-sets
OS-SCH-HNT
os-server-diagnostics
os-shelve
OS-SRV-USG
os-tenant-networks
os-user-data
os-user-quotas
public
server-action-changepassword
server-action-confirmresize
server-action-createimage
server-action-reboot
server-action-rebuild
server-action-rebuild-resp
server-action-rebuild
server-action-resize
server-action-revertresize
server-get-resp
server-ips-network-resp
server-ips-resp
server-metadata-all-req
server-metadata-all-resp
server-metadata-req
server-metadata-resp
server-post-req
server-post-resp
server-put-req2
server-put-req-ad
server-put-req
server-put-resp-ad
server-put-resp
servers-details-resp
servers-list-resp
versions-get-resp

Editor

I know on the wiki the recommended editor is Oxygen, but because of the extra trouble I have to go through to obtain the license (although Oxygen company sponsor OpenStack with some 6-month licenses, it’s still a pain to have to worry about the license every half a year) and not to mention that I have to learn different short cut key combinations and UI. I have decided to use the editors I used before, hence already installed.

Sublime is the overall winner.

About the license:

Sublime Text 2 may be downloaded and evaluated for free, however a license must be purchased for continued use. There is currently no enforced time limit for the evaluation.

The only problem I have encountered with the other editors (Context, Notepad++) is tab/white space. Their automatic indentation, the number of white spaces inserted when hit “tab” is just not right. Jenkins (the auto code testing & niceness checking system) shows a difference indentation to my editor, as it complains.

Well, there may be a way to configure the editors to add space as Jenkins expected, but I’m too lazy to find out, since all the above mentioned editors just happen to be already installed on one of the computer available to me.

Nono is another editor I’d like to mention, I only use it for small editing over the terminal on the remote server. I have heard about the powerful Vi, but the insertion mode & all the commands have to learn makes me nervous, guess there is not enough reason to force me into using it. Like I can still remember there was a course I took teaching you Agda, the only editor support it is Emacs (at that time at least).

Follow

Get every new post delivered to your Inbox.