Commit cf99ff01 by akhiljain23

Updating docs

>> Changed project name from plasma to valence in docs.
>> Fixed docstring indentation.
>> Added features docs.
>> Added driver docs.
>> Hidden licence headers from html files.

Depends-On: Ia02bc00ad168b3c3d01ef6ca9696d43996091070
Change-Id: I1fa382d566165f5e76c84ad864024c0f546ef74c
parent 47777b21
......@@ -2,100 +2,79 @@
Openstack Valence Project
=========================
********
Overview
********
Valence is a service for lifecycle management of pooled bare-metal hardware
infrastructure such as Intel(R) Rack Scale architecture which uses Redfish(TM)
as one of the management protocols.
infrastructure. The concept of pooled storage (SSDs or nvmE) disaggregated
from compute nodes and network disaggregated from compute and storage
provides the flexibility to compose and uses as and when the cloud requires
more server resources. Valence provides the capability "compose" hardware nodes
and release resources as needed by the overcloud.
Valence supports Redfish as default management protocol to communicate
to hardware. It supports Rack Scale Design (RSD), Open architecture for management
of disaggregated server hardware resources, which is standardized in Redfish.
Valence also provides capability to manage other vendors disaggregated hardware
using their respective drivers other than Redfish.
:Free software: Apache license
:Wiki: https://wiki.openstack.org/wiki/Valence
:Source: http://git.openstack.org/cgit/openstack/valence
:Bugs: https://bugs.launchpad.net/openstack-valence
.. _valence-features:
=========================
Download and Installation
=========================
The following steps capture how to install valence. All installation steps
require super user permissions.
**************************
Database etcd installation
**************************
Single node installation reference: https://github.com/coreos/etcd/releases
Distributed installation reference: https://github.com/coreos/etcd/blob/master/Documentation/op-guide/clustering.md
For development, single node installation is recommended practice.
********************
Valence installation
********************
1. Install software dependencies
``$ sudo apt-get install git python-pip python-dev build-essential``
2. Clone the Valence code from git repo.
``$ git clone https://git.openstack.org/openstack/valence``
3. Execute the 'install_valence.sh' file present in the Valence root directory.
The install script will automatically install the dependencies listed in the
requirements.txt file.
******************
Feature Highlights
******************
``$ sudo bash install_valence.sh``
Valence provides the following capabilities for managing disaggregated hardware resources.
4. Check the values in valence.conf located at /etc/valence/valence.conf
.. _multi-podm-support:
``set the ip/credentials of podm for which this Valence will interact``
Multiple Podmanager Support
---------------------------
5. Check the PYTHON_HOME and other variables in /etc/init/valence.conf
Valence provides the capability to manage multiple podmanagers.
Each podmanager can be configured to use different driver. By default,
``redfishv1`` driver is used.
6. Initialize etcd database
Currently supported drivers in Valence are:
#. :ref:`redfishv1-driver`
#. :ref:`expether-driver`
``$ valence-db-manager init``
Future work include redfish v2 driver support. Other vendors also could implement
their own driver to manage their hardware.
Note: The TypeError exception "TypeError: NoneType object is not callable"
is caused by known python-etcd bug, which will not impact this db init
functionality.
https://github.com/jplana/python-etcd/issues/190
This provides uniform interface to manage all the disaggregated hardware in datacentre
supporting different protocols.
7. Start valence service
.. _device-orchestration:
``$ sudo service valence start``
Device Orchestration framework
------------------------------
8. Logs are located at /var/logs/valence/
Valence provides support for the dynamic management of pooled resources like storage,
network and other pci devices which can be connected on demand to a composed node,
giving user the ability to attach or detach the devices dynamically based on workload.
****************
GUI installation
****************
Please refer to the installation steps in the ui/README file.
.. _ironic-provision-driver:
Ironic provision driver
-----------------------
**********
Components
**********
Valence supports ``ironic`` provision driver to be able to register the valence composed
node to ironic. Once the node is registered further operations of provisioning/managing
could be directly performed using Ironic_.
Valence follows the typical OpenStack project setup. The components are listed
below:
For more details on Ironic usage:
`Ironic API guide <https://developer.openstack.org/api-ref/baremetal/>`_.
valence-api
-----------
A python based daemon based on Flask framework to expose Valence REST APIs.
The api service communicates to the PODM through REST interface using Redfish(TM) specification.
For adding new api please refer https://github.com/openstack/valence/blob/master/doc/source/developer-guide/add_new_api.rst
valence-ui
----------
valence-ui provides a Web-based GUI interface that can be used to explore
Rack Scale Design (RSD) artifacts and compose/disassemble nodes.
valence-ui is implemented using Node.js runtime environment and hosted through apache.
valence-ui makes us of React.js javascript libaray and invoke Valence REST APIs through ajax REST calls.
For more features please refer the Valence blueprints_ for supported and
in-the-pipeline features.
========
Features
========
Please refer the Valence blueprints for supported and in-the-pipeline features.
``https://blueprints.launchpad.net/openstack-valence``
.. _blueprints: https://blueprints.launchpad.net/openstack-valence
.. _Ironic: https://docs.openstack.org/ironic/latest/
......@@ -15,3 +15,4 @@ This is a reference for the Openstack Valence API
.. include:: valence-api-v1-system.inc
.. include:: valence-api-v1-nodes.inc
.. include:: valence-api-v1-pods.inc
.. include:: valence-api-v1-devices.inc
{
"created_at": "2018-04-09 05:57:55 UTC",
"extra": {
"device_name": "PCIe Data Center SSD",
"vendor_name": "Intel Corporation"
},
"node_id": null,
"podm_id": "64420559-0f44-4ca1-82c2-ac9a26f6788b",
"pooled_group_id": "4093",
"properties": {
"device_id": "0x000000000000",
"mac_address": "00:11:22:33:44:55",
"model": "Generation-X"
},
"resource_uri": "devices/0x000000000000",
"state": "free",
"type": "SSD",
"updated_at": "2018-04-09 05:57:55 UTC",
"uuid": "95a6b68f-9c3a-47f8-b93b-c160c940d7b7"
}
[
{
"node_id": "0x000000000000",
"podm_id": "11111111-1111-1111-1111-111111111111",
"pooled_group_id": "2223",
"resource_uri": "devices/2x222222222222",
"state": "allocated",
"type": "SSD",
"uuid": "95a6b68f-9c3a-47f8-b93b-c160c940d7b7"
},
{
"node_id": null,
"podm_id": "64420559-0f44-4ca1-82c2-ac9a26f6788b",
"pooled_group_id": "4093",
"resource_uri": "devices/3x333333333333",
"state": "free",
"type": "NIC",
"uuid": "603ba114-574e-48d3-afa2-9fd8d98fe0d2"
}
]
[
{
"podm_id": "64420559-0f44-4ca1-82c2-ac9a26f6788b",
"status": "SUCCESS"
}
]
{
"url": "http://0.0.0.0:00000/server",
"name": "podm1",
"driver": "driver_name",
"authentication": [{
"type": "basic",
"auth_items": {
"username": "admin",
"password": "password"
}
}]
}
{
"authentication": [
{
"auth_items": {
"password": "***",
"username": "admin"
},
"type": "basic"
}
],
"created_at": "2018-04-16 10:01:27 UTC",
"driver": "driver_name",
"name": "podm1",
"status": "Online",
"updated_at": "2018-04-16 10:01:27 UTC",
"url": "http://0.0.0.0:00000/server",
"uuid": "776a4f7b-195d-49c9-9b52-b25760d85158"
}
.. -*- rst -*-
=======
Devices
=======
List, Searching and Sync of devices are done through the ``/v1/devices``
List Devices
============
.. rest_method:: GET /v1/devices/
Return a list of Devices.
Some filtering is possible by passing in filters with the request.
.. NOTE::
Request format: GET /v1/devices?{filters}
where filters can be combination of keys: node_id, podm_id, state,
type, pooled_group_id, resource_uri.
For example: GET /v1/devices?node_id=0x000000&type=SSD
By default, this query will return all devices with uuid, podm_id,
type, state, node_id, resource_uri and pooled_group_id.
Normal response codes: 200
Error response codes: badRequest(400), unauthorized(401), forbidden(403)
Request
-------
Response
--------
.. rest_parameters:: parameters.yaml
- node_id: node_index
- podm_id: podm_uuid
- pooled_group_id: device_group_id
- resource_uri: device_uri
- state: device_state
- type: device_type
- uuid: device_uuid
**Example list of Devices:**
.. literalinclude:: mockup/devices-list-response.json
:language: javascript
Display Device Details
======================
.. rest_method:: GET /v1/devices/{device_id}
Shows details for a Device.
This will return the full representation of the device.
Normal response codes: 200
Error response codes: badRequest(400), unauthorized(401), forbidden(403)
Request
-------
.. rest_parameters:: parameters.yaml
- id: device_id
Response
--------
.. rest_parameters:: parameters.yaml
- created_at: created_at
- extra: device_details
- node_id: node_index
- podm_id: podm_uuid
- pooled_group_id: device_group_id,
- properties: device_properties
- resource_uri: device_uri
- state: device_state
- type: device_type
- updated_at: updated_at
- uuid: device_uuid
**Example JSON representation of a Device:**
.. literalinclude:: mockup/devices-get-response.json
:language: javascript
Sync Devices
============
.. rest_method:: POST /v1/devices/sync
Sync all devices managed with podmanager.
Synchronization of devices connected to one podmanager is possible by
passing ``podm_id`` in the body of POST request.
By default it will sync devices connected to all podmanagers one by one.
Normal response codes: 200
Error response codes: badRequest(400), unauthorized(401), forbidden(403)
Request
-------
Response
--------
.. rest_parameters:: parameters.yaml
- podm_id: podm_uuid
- status: sync_status
**Example JSON representation of a Sync response:**
.. literalinclude:: mockup/devices-sync-response.json
:language: javascript
......@@ -4,9 +4,59 @@
Pod managers
==============
Listing, searching of Pod Manager resources is done through the ``/v1/pod_managers``
Creating, Listing, searching of Pod Manager resources is done through the
``/v1/pod_managers``
Send feedback to Valence team or [chester.kuo@gmail.com]
Create Pod manager
==================
.. rest_method:: POST /v1/pod_manager
Creates a new Pod Manager with specific URL, name, authentication credentials
and driver(default driver: ``redfishv1``).
Normal response codes: 200
Error response codes: badRequest(400), unauthorized(401), forbidden(403)
Request
-------
.. rest_parameters:: parameters.yaml
- name: podmanager_name
- url: pod_conn_url
- driver: podm_driver
- authentication: auth_details
**Example Pod Manager creation request:**
.. literalinclude:: mockup/pod-manager-create-request.json
:language: javascript
Response
--------
The response will contain the complete pod uuid and name record
The list and example below are representative of the response as of API
.. rest_parameters:: parameters.yaml
- name: podmanager_name
- url: pod_conn_url
- driver: podm_driver
- authentication: auth_details
- created_at: created_at
- updated_at: updated_at
- uuid: pod_uuid
**Example JSON representation of a Pod Manager:**
.. literalinclude:: mockup/pod-manager-create-response.json
:language: javascript
List Pod Manager
=================
......@@ -97,4 +147,3 @@ Request
.. rest_parameters:: parameters.yaml
- pod_uuid: pod_uuid
......@@ -8,7 +8,7 @@ Concepts
========
In order to bring new features to users over time, the Valence API
supports versioning. There are two kinds of versions in Ironic.
supports versioning. There are two kinds of versions in Valence.
- ''major versions'', which have dedicated urls.
- ''microversions'', which can be requested through the use of the
......@@ -16,7 +16,7 @@ supports versioning. There are two kinds of versions in Ironic.
The Version APIs work differently from other APIs as they do not require authentication.
Alll API requests support the ``OpenStack-API-Version`` header.
All API requests support the ``OpenStack-API-Version`` header.
This header SHALL be supplied with every request.
This help to provide backwards compatibility as we introduced new features in the server.
......
{
"name" : "OpenStack Plasma API",
"description" : "Plasma is an OpenStack project which aims to provide node composition based on redfish API.",
"name" : "OpenStack Valence API",
"description" : "Valence is an OpenStack project which aims to provide node composition based on redfish API.",
"default_version" : {
"status" : "CURRENT",
"version" : "1.1",
......
......@@ -8,7 +8,7 @@
{
"rel" : "describedby",
"type" : "text/html",
"href" : "http://docs.openstack.org/developer/plasma/dev/api-spec-v1.html"
"href" : "http://docs.openstack.org/developer/valence/dev/api-spec-v1.html"
}
],
"nodes" : [
......@@ -43,7 +43,7 @@
],
"media_types" : [
{
"type" : "application/vnd.openstack.plasma.v1+json",
"type" : "application/vnd.openstack.valence.v1+json",
"base" : "application/json"
}
]
......
============
Contributing
============
.. include:: ../../CONTRIBUTING.rst
\ No newline at end of file
..
Copyright (c) 2018 NEC, Corp.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _valence-contribution:
============
Contributing
============
.. include:: ../../../CONTRIBUTING.rst
.. _add_new_api:
..
Copyright 2016 Intel Corporation
All Rights Reserved.
......@@ -14,6 +14,8 @@
License for the specific language governing permissions and limitations
under the License.
.. _add-new-api:
=================================
Add a new API endpoint in Valence
=================================
......@@ -24,7 +26,7 @@ through the `readme.rst`).
Interacting with Valence
-------------------------
------------------------
Before starting modification/adding a new functionality in Valence, it is suggested
to check that the valence has been setup properly. This could be done by making some
......
.. _valence_add_new_developer_guide:
..
Copyright 2016 Intel Corporation
All Rights Reserved.
......@@ -14,6 +14,8 @@
License for the specific language governing permissions and limitations
under the License.
.. _valence_add_new_developer_guide:
============================
Add a developer guide doc
============================
......
.. _valence_functional_testcase:
..
Copyright 2016 Intel Corporation
All Rights Reserved.
......@@ -14,6 +14,8 @@
License for the specific language governing permissions and limitations
under the License.
.. _valence_functional_testcase:
==========================
Add a Functional Test case
==========================
......@@ -34,10 +36,10 @@ Tests scripts are located in `<valence_root>/valence/tests
Implementing a Test Case
------------------------
Consider implementing an functional testcase for our /example(add_new_api_) API
Consider implementing an functional testcase for our /example(:ref:`add-new-api`) API
The characteristics of /example(add_new_api_) API
The characteristics of /example(:ref:`add-new-api`) API
* REST Method : GET
* Parameters : Nil
......@@ -81,3 +83,4 @@ running the below command from the valence root directory(where tox.ini is prese
.. code-block:: bash
$ tox -e pep8,py27
.. _valence_unit_testcase:
..
Copyright 2016 Intel Corporation
All Rights Reserved.
......@@ -14,9 +14,11 @@
License for the specific language governing permissions and limitations
under the License.
======================
.. _valence_unit_testcase:
====================
Add a Unit Test case
======================
====================
Getting used to writing testing code and running this code in parallel is considered
a good workflow.
......
..
Copyright (c) 2017 NEC, Corp.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _valence-components:
==========
Components
==========
Valence follows the typical OpenStack project setup. The components are listed
below:
valence-api
-----------
A python based daemon based on Flask framework to expose Valence REST APIs.
The api service communicates to the PODM through REST interface using Redfish(TM) specification.
For adding new api please refer https://github.com/openstack/valence/blob/master/doc/source/developer-guide/add_new_api.rst
valence-ui
----------
valence-ui provides a Web-based GUI interface that can be used to explore
Rack Scale Design (RSD) artifacts and compose/disassemble nodes.
valence-ui is implemented using Node.js runtime environment and hosted through apache.
valence-ui makes us of React.js javascript library and invoke Valence REST APIs through ajax REST calls.
.. _valence_db_development:
..
Copyright 2016 Intel Corporation
All Rights Reserved.
......@@ -14,6 +14,8 @@
License for the specific language governing permissions and limitations
under the License.
.. _valence_db_development:
==============
DB Development
==============
......
.. _valence_deploy:
..
Copyright 2016 Intel Corporation
All Rights Reserved.
......@@ -15,6 +14,8 @@
License for the specific language governing permissions and limitations
under the License.
.. _valence_deploy:
======================
Deploy Valence Project
======================
......
..
Copyright (c) 2018 NEC, Corp.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _expether-driver:
===============
ExpEther Driver
===============
Overview
========
The ``ExpEther`` driver enables management of ExpEther Hardware through
Openstack Valence. It relies on EEM REST API to communicate to hardware.
For more details on hardware, please refer ExpEther_.
Prerequisites
=============
ExpEtherManager(EEM) should be configured and reachable from the controller.
It can be installed on node different from Openstack Controller.
``hwdata`` package is used to fetch PCI device details (vendor name,
device name) in human readable format. Retrieval of details is only
supported for 40g devices. If package is not available, the same
are shown as None.
For ubuntu, package can be installed using following command::
$ sudo apt-get install hwdata
Supported Functionalities
=========================
* To be able to use the driver, first the podmanager needs to be created
with connection information of the EEM service.
The details needed for the same are:
- EEM REST URL : http://<ip>:<port>/eem
- Authentication Information (username/password)
- driver : 'expether'
To register the same please refer :ref:`example <create-pod-manager>`
* Once the podmanager is created, the same will be used for subsequent requests
to communicate with hardware.
* Using ``expether`` driver the following functionalities are supported in Valence:
#. Node Management, Server can be composed as per user requested flavor or
the existing node can be made to be managed by Valence.
Currently, for composing node, only supported flavor keys are ``pci_device``.
Please refer :ref:`sample request <compose-node>`
#. All the IO devices would be automatically discovered by the podmanager and
would be managed by Valence. Also supports allocation of PCI devices from the
resource pool to node as per workload demand.
EE node managed by valence can be assigned from the pool of devices as per
user request and the same will be released to pool once the workload decreases.
#. Periodic discovery of all resources from podmanager is supported,
which registers any newly connected resources to valence and keeps the status sync
with podmanager and valence DB.
#. Nodes managed by Valence can be made available to Ironic, for further provisioning using
ironic provision driver supported in valence. For more details
refer :ref:`ironic-provision-driver` and :ref:`sample request <node-register>`
Sample Requests
===============
.. _create-pod-manager:
Create a pod_manager
--------------------
Create a new podmanager connecting to ExpressEther Manager.
Sample request:
.. code-block:: bash
curl -i -X POST "http://localhost:8181/v1/pod_managers" \
-d '{"url": "http://<ip>:<port>/eem", "name": "podm1", \
"driver": "expether", "authentication": [{"type": "basic", \
"auth_items": {"username": "xxx", "password": "xxxx"}}]}' \
-H "Accept:application/json" -H "Content-Type:application/json"
Response:
.. code-block:: json