mirror of
https://dev.iopsys.eu/bbf/bbfdm.git
synced 2026-01-27 17:37:17 +01:00
Update Documentation
This commit is contained in:
Amin Ben Romdhane
parent
af2b0c0203
commit
5de7633288
7 changed files with 427 additions and 303 deletions
29
README.md
29
README.md
|
|
@ -2,13 +2,15 @@
|
|||
|
||||
`bbfdm` is a suite to provide TR181 datamodel backend for Higher layer management protocols like [TR-069/CWMP](https://cwmp-data-models.broadband-forum.org/) or [TR-369/USP](https://usp.technology/). It is designed in a hardware agnostic way and provides the available datamodel parameters over ubus on the northbound interface and creates the datamodel mapping based on uci and ubus on southbound interface.
|
||||
|
||||
`bbfdm` has three main components:
|
||||
`bbfdm` has five main components:
|
||||
|
||||
| Component | Description |
|
||||
| ------------ | ------------------------------------------------- |
|
||||
| bbfdmd | A daemon to expose data model objects over ubus |
|
||||
| libbbfdm-api | API library to create and parse datamodel tree |
|
||||
| libbbfdm | Minimal TR181 datamodel definition |
|
||||
| bbfdmd | A daemon to expose data model objects over ubus |
|
||||
| dm-service | A daemon to expose data model objects as micro-service over ubus |
|
||||
| libbbfdm-api | API library to create and parse datamodel tree |
|
||||
| libbbfdm-ubus | API library to expose datamodel over ubus |
|
||||
| libbbfdm | A static library that contains the core data model of TR181 |
|
||||
|
||||
|
||||
## Directory Structure
|
||||
|
|
@ -18,10 +20,12 @@
|
|||
```bash
|
||||
├── bbfdmd -- This directory contains daemon code to expose the datamodel tree on northbound
|
||||
│ └── ubus - Daemon to expose datamodel over ubus
|
||||
├── dm-service -- This directory contains daemon code to expose the datamodel tree as micro-service
|
||||
├── docs -- More detailed explanation of datamodel and user guide
|
||||
├── gitlab-ci -- Used for CI/CD pipeline test
|
||||
├── libbbfdm -- Minimal TR181 datamodel implementation
|
||||
├── libbbfdm -- Minimal TR181 core datamodel implementation
|
||||
├── libbbfdm-api -- API library to create datamodel definition and parse the datamodel definition to form a datamodel tree
|
||||
├── libbbfdm-ubus -- API library to expose datamodel over ubus
|
||||
├── tools -- Tools to convert xml datamodel definition to json, generate c code and many more
|
||||
└── utilities -- Small helper utilities to complete/optimize the datamodel deployment
|
||||
```
|
||||
|
|
@ -38,9 +42,6 @@
|
|||
### Datamodel related topics
|
||||
|
||||
* [Design for firmware activation](./docs/guide/libbbfdm_DeviceInfo_FirmwareImage.md)
|
||||
* [Different Network Deployment Scenarios using Datamodels](./docs/guide/network_depoyment_scenarios.md)
|
||||
* [GRE datamodel details ](./docs/guide/libbbfdm_GRE.md)
|
||||
* [IP datamodel details](./docs/guide/libbbfdm_IP_Interface.md)
|
||||
|
||||
### Compilation helper utilities
|
||||
|
||||
|
|
@ -66,8 +67,8 @@ To successfully build bbfdmd, following libraries are needed:
|
|||
| libubus | https://git.openwrt.org/project/ubus.git | LGPL 2.1 |
|
||||
| libjson-c | https://s3.amazonaws.com/json-c_releases | MIT |
|
||||
| libbbfdm-api | https://dev.iopsys.eu/bbf/bbfdm.git | BSD-3 |
|
||||
| libbbfdm-ubus | https://dev.iopsys.eu/bbf/bbfdm.git | BSD-3 |
|
||||
| libbbfdm | https://dev.iopsys.eu/bbf/bbfdm.git | BSD-3 |
|
||||
| jq | https://github.com/stedolan/jq.git | BSD |
|
||||
|
||||
|
||||
### Run-Time Dependencies
|
||||
|
|
@ -78,5 +79,13 @@ In order to run the `bbfdmd`, following dependencies are needed to be running/av
|
|||
| ------------ | ---------------------------------------- | -------- |
|
||||
| ubusd | https://git.openwrt.org/project/ubus.git | LGPL 2.1 |
|
||||
| libbbfdm-api | https://dev.iopsys.eu/bbf/bbfdm.git | BSD-3 |
|
||||
| libbbfdm-ubus | https://dev.iopsys.eu/bbf/bbfdm.git | BSD-3 |
|
||||
| libbbfdm | https://dev.iopsys.eu/bbf/bbfdm.git | BSD-3 |
|
||||
| jq | https://github.com/stedolan/jq.git | BSD |
|
||||
|
||||
In order to run the `dm-service`, following dependencies are needed to be running/available before `dm-service`.
|
||||
|
||||
| Dependency | Link | License |
|
||||
| ------------ | ---------------------------------------- | -------- |
|
||||
| ubusd | https://git.openwrt.org/project/ubus.git | LGPL 2.1 |
|
||||
| libbbfdm-api | https://dev.iopsys.eu/bbf/bbfdm.git | BSD-3 |
|
||||
| libbbfdm-ubus | https://dev.iopsys.eu/bbf/bbfdm.git | BSD-3 |
|
||||
|
|
|
|||
|
|
@ -20,11 +20,11 @@
|
|||
"description": "Enabled debug logging"
|
||||
},
|
||||
{
|
||||
"name": "sock",
|
||||
"type": "string",
|
||||
"name": "loglevel",
|
||||
"type": "unsigned integer",
|
||||
"required": "no",
|
||||
"default": "",
|
||||
"description": "Path for ubus socket to register bbfdmd services"
|
||||
"default": "3",
|
||||
"description": "Log verbosity value as per standard syslog"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
|
|
|||
|
|
@ -69,19 +69,19 @@
|
|||
</tr>
|
||||
<tr>
|
||||
<td class="td_row_even">
|
||||
<div class="td_row_even">sock</div>
|
||||
<div class="td_row_even">loglevel</div>
|
||||
</td>
|
||||
<td class="td_row_even">
|
||||
<div class="td_row_even">string</div>
|
||||
<div class="td_row_even">unsigned integer</div>
|
||||
</td>
|
||||
<td class="td_row_even">
|
||||
<div class="td_row_even">no</div>
|
||||
</td>
|
||||
<td class="td_row_even">
|
||||
<div class="td_row_even"></div>
|
||||
<div class="td_row_even">3</div>
|
||||
</td>
|
||||
<td class="td_row_even">
|
||||
<div class="td_row_even">Path for ubus socket to register bbfdmd services</div>
|
||||
<div class="td_row_even">Log verbosity value as per standard syslog</div>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
|
|
|
|||
|
|
@ -6,42 +6,11 @@
|
|||
|
||||
## Concepts and Workflow
|
||||
|
||||
`bbfdmd` daemon gets started by `/etc/init.d/bbfdmd` service, bbfdmd init script reads the input from `bbfdm` uci file and then it generates the input for `bbfdmd` daemon in json format in `/tmp/bbfdm/input.json`
|
||||
`bbfdmd` daemon gets started by `/etc/init.d/bbfdmd` service, bbfdmd init script reads the input from `bbfdm` uci file and then it starts using the APIs provided by `libbbfdm-ubus` and `libbbfdm-api`.
|
||||
|
||||
```json input.json
|
||||
{
|
||||
"daemon": {
|
||||
"config": {
|
||||
},
|
||||
"input": {
|
||||
"plugin_dir": "/usr/share/bbfdm/plugins"
|
||||
},
|
||||
"output": {
|
||||
"type": "UBUS",
|
||||
"name": "bbfdm"
|
||||
}
|
||||
},
|
||||
"cli": {
|
||||
"config": {
|
||||
"proto": "both"
|
||||
},
|
||||
"input": {
|
||||
"type": "UBUS",
|
||||
"name": "bbfdm"
|
||||
},
|
||||
"output": {
|
||||
"type": "CLI"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
`bbfdmd` daemon use `libbbfdm-api` library to traverse the core datamodel tree added by static library `libbbfdm` or other plugins defined in `/usr/share/bbfdm/plugins`.
|
||||
|
||||
When `bbfdmd` starts it parses the `input.json` file to check the different configurations, like
|
||||
- Name of the TR181 datamodel definition file (daemon.input.name)
|
||||
- Location of datamodel definition loaded as plugins (daemon.input.plugin_dir
|
||||
- Name of the ubus object it has to register (daemon.output.name)
|
||||
|
||||
`bbfdmd` daemon use `libbbfdm-api` library to traverse the datamodel tree added by 'libbbfdm' or other plugins added in predefined plugin path(daemon.input.plugin_dir).
|
||||
`bbfdmd` daemon use `libbbfdm-ubus` library to expose datamodel over ubus.
|
||||
|
||||
When a ubus method is called it first fills `bbfdm_data_t` structure with the necessary information, then proceeds the `Get/Set/Operate/Add/Del` operation based on that information.
|
||||
|
||||
|
|
@ -49,36 +18,6 @@ To load the datamodel definitions from a DotSO file, it looks for a 'tDynamicObj
|
|||
|
||||
In short, it covers/supports all methods introduced in `TR-069` and `TR-369` by using the `bbf_entry_method` API from `libbbfdm-api` with the different methods and the existing data-model available with `libbbfdm`.
|
||||
|
||||
`bbfdmd` daemon can also be used to expose a module specific datamodel or sub-tree, by using datamodel micro-services. Datamodel micro-service is nothing but another `bbfdmd` instance running with smaller data sub-set, micro-service can be identified in the runtime by running a `ps` command
|
||||
|
||||
```bash
|
||||
# ps|grep bbf
|
||||
3804 root 6968 S {dm_bridgemngr} /usr/sbin/bbfdmd -m bridgemngr
|
||||
3805 root 7064 S {dm_bulkdata} /usr/sbin/bbfdmd -m bulkdata
|
||||
3806 root 6900 S {dm_ddnsmngr} /usr/sbin/bbfdmd -m ddnsmngr
|
||||
3807 root 6996 S {dm_dhcpmngr} /usr/sbin/bbfdmd -m dhcpmngr
|
||||
```
|
||||
|
||||
A `bbfdmd` instance with -m input means its running a module(sub-tree) datamodel. These micro-services exposed their own ubus objects
|
||||
|
||||
```bash
|
||||
# ubus list bbfdm.*
|
||||
bbfdm.Bridging
|
||||
bbfdm.BulkData
|
||||
bbfdm.DHCPv4_DHCPv6
|
||||
bbfdm.DNS
|
||||
```
|
||||
|
||||
When a datamodel started as micro-service, it looks/waits for `bbfdm` ubus object(added by main bbfdmd process), once its available micro-service resister with the main `bbfdmd` process by calling it 'service' method
|
||||
|
||||
```bash
|
||||
# ubus -v list bbfdm
|
||||
'bbfdm' @e970413c
|
||||
"service":{"cmd":"String","name":"String","parent_dm":"String","objects":"Array"}
|
||||
```
|
||||
|
||||
More details about micro-services covered in a dedicated section.
|
||||
|
||||
> Note: In general, bbfdmd does not reload the services after updating the configs, higher layer applications (i.e. icwmp, obuspa) usages `bbf.config` to apply the configs and reloads the services, please check `bbf.config` documentation for more details.
|
||||
|
||||
## Input and output Schema(s)
|
||||
|
|
@ -91,27 +30,6 @@ More details about micro-services covered in a dedicated section.
|
|||
- [bbfdm ubus guide](../api/ubus/bbfdm.md)
|
||||
- [bbfdm ubus schema](../api/ubus/bbfdm.json)
|
||||
|
||||
`bbfmdd` can also be used to expose sub tree using different ubus object by using datamodel micro-service, ubus guide and schema for datamodel micro-services
|
||||
|
||||
- [datamodel micro-service](../api/ubus/bbfdm_micro_service.md)
|
||||
- [datamodel micro-service schema](../api/ubus/bbfdm_micro_service.json)
|
||||
|
||||
Apart from these, datamodel micro-services can also be configured by updating their input files directly at the runtime, micro-service input files present in `/etc/bbfdm/micro_services/` in CPE.
|
||||
|
||||
Datamodel micro-service has a similar input file as of main bbfdmd
|
||||
|
||||
```json
|
||||
{
|
||||
"daemon": {
|
||||
"enable": "1",
|
||||
"service_name": "sshmngr",
|
||||
"config": {
|
||||
"loglevel": "1"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Ubus methods
|
||||
|
||||
Following are the ubus methods exposed by `bbfdmd` main process:
|
||||
|
|
@ -130,20 +48,6 @@ Following are the ubus methods exposed by `bbfdmd` main process:
|
|||
"notify_event":{"name":"String","input":"Array"}
|
||||
```
|
||||
|
||||
Each datamodel micro-service expose their own ubus object, which is slightly different from main `bbfdm` ubus object, following is an example of ubus methods exposed by datamodel micro-services.
|
||||
|
||||
```bash
|
||||
# ubus -v list bbfdm.SSH
|
||||
'bbfdm.SSH' @bb8a66da
|
||||
"get":{"path":"String","paths":"Array","maxdepth":"Integer","optional":"Table"}
|
||||
"schema":{"path":"String","paths":"Array","first_level":"Boolean","optional":"Table"}
|
||||
"instances":{"path":"String","paths":"Array","first_level":"Boolean","optional":"Table"}
|
||||
"set":{"path":"String","value":"String","obj_path":"Table","optional":"Table"}
|
||||
"operate":{"command":"String","command_key":"String","input":"Table","optional":"Table"}
|
||||
"add":{"path":"String","obj_path":"Table","optional":"Table"}
|
||||
"del":{"path":"String","paths":"Array","optional":"Table"}
|
||||
```
|
||||
|
||||
> Note1: `optional` table are present in all methods and it supports below options:
|
||||
|
||||
```console
|
||||
|
|
@ -160,178 +64,19 @@ Each datamodel micro-service expose their own ubus object, which is slightly dif
|
|||
|
||||
> Note4: Check ubus schema document for more details
|
||||
|
||||
### Fault handling
|
||||
|
||||
To indicate a fault and source of fault `bbfdmd` provides `fault` along with `fault_msg` in the response in case of faults, which then handled by higher layer applications (i.e icwmp, obuspa).
|
||||
|
||||
This provides a clear inside on the root cause of the fault, and based on `fault_msg` it's easily to understand what the issue is and how to fix it and find out the limitations(if there are any on the device).
|
||||
|
||||
#### Example(s)
|
||||
|
||||
1. The requested value is correct as per TR181 standard, but there is a limitation in the device.
|
||||
|
||||
```console
|
||||
root@iopsys:~# ubus call bbfdm set '{"path":"Device.Firewall.Config", "value":"High"}'
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"path": "Device.Firewall.Config",
|
||||
"fault": 9007,
|
||||
"fault_msg": "The current Firewall implementation supports only 'Advanced' config."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
2. The requested value is outside the allowed range.
|
||||
|
||||
```console
|
||||
root@iopsys:~# ubus call bbfdm set '{"path":"Device.Firewall.Chain.1.Rule.9.DestPort", "value":"123456"}'
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"path": "Device.Firewall.Chain.1.Rule.9.DestPort",
|
||||
"fault": 9007,
|
||||
"fault_msg": "'123456' value is not within range (min: '-1' max: '65535')"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
3. Some arguments should be defined to perform the requested operation.
|
||||
|
||||
```console
|
||||
root@iopsys:~# ubus call bbfdm operate '{"command":"Device.IP.Diagnostics.IPPing()", "command_key":"ipping_test", "input":{}}'
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"path": "Device.IP.Diagnostics.IPPing()",
|
||||
"data": "ipping_test",
|
||||
"fault": 7004,
|
||||
"fault_msg": "IPPing: 'Host' input should be defined"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
4. The path parameter value must start with 'Device.'. The command below doesn't have Device before path "Users.User."
|
||||
|
||||
```console
|
||||
root@iopsys:~# ubus call bbfdm get '{"path":"Users.User.", "optional": {"format":"raw", "proto":"usp"}}'
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"path": "Users.User.",
|
||||
"fault": 7026,
|
||||
"fault_msg": "Path is not present in the data model schema"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
These fault messages defined in datamodel handlers, user can add such fault message using `bbfdm_set_fault_message` libbbfdm-api's API, if no specific fault message defined for particular obj/param, datamodel returns standard error messages that are defined in CWMP and USP protocols as the fault message value.
|
||||
|
||||
##### Errors Codes
|
||||
|
||||
| Error Code | Meaning |
|
||||
|------------|--------------------------------------------------------------|
|
||||
| 7003 | Message failed due to an internal error. |
|
||||
| 7004 | Message failed due to invalid values in the request elements and/or failure to update one or more parameters during Add or Set requests. |
|
||||
| 7005 | Message failed due to memory or processing limitations. |
|
||||
| 7008 | Requested path was invalid or a reference was invalid. |
|
||||
| 7010 | Requested Path Name associated with this ParamError did not match any instantiated parameters. |
|
||||
| 7011 | Unable to convert string value to correct data type. |
|
||||
| 7012 | Out of range or invalid enumeration. |
|
||||
| 7022 | Command failed to operate. |
|
||||
| 7026 | Path is not present in the data model schema. |
|
||||
|
||||
|
||||
### Parallel calls over Ubus
|
||||
|
||||
Some datamodel operations takes less time to execute compared to other, like
|
||||
- Get on sub-set of datamodel or an individual datamodel parameter takes very less, where as
|
||||
- Get on complete Device. and Async operate commands takes much longer
|
||||
|
||||
executing/serializing operations simplifies the code from developer perspective, but its not suitable for deployments. To make it suitable `bbfdmd` support parallel calls.
|
||||
|
||||
- All datamodel `operate` commands are running in parallel
|
||||
|
||||
example(s):
|
||||
|
||||
```console
|
||||
root@iopsys:~# time ubus call bbfdm get '{"path":"Device."}' >/dev/null &
|
||||
root@iopsys:~# time ubus call bbfdm get '{"path":"Device.Users."}' >/dev/null
|
||||
real 0m 0.07s
|
||||
user 0m 0.00s
|
||||
sys 0m 0.00s
|
||||
root@iopsys:~#
|
||||
real 0m 1.86s
|
||||
user 0m 0.05s
|
||||
sys 0m 0.00s
|
||||
|
||||
[1]+ Done time ubus call bbfdm get "{\"path\":\"Device.\"}" >/dev/null
|
||||
root@iopsys:~#
|
||||
```
|
||||
|
||||
## Datamodel micro-services
|
||||
|
||||
Datamodel micro-service is nothing but running a partial datamodel sub-set with another instance of `bbfdmd` binary.
|
||||
|
||||
Benefit:
|
||||
- Instead of having a huge datamodel tree, it split the tree based on modules, which reduce the cost of operation on the tree
|
||||
- Resolves one point failure, if any datamodel micro-service not working correctly it only affects that specific module, not the rest of the tree
|
||||
- Moving datamodel code to module repo, brings the service layer and datamodel layer closer, which lowers the possibilities of wrong mapping
|
||||
- Possible to use lightweight `bbfdmd` inside containers with json plugin for Container service management
|
||||
- Possible to impose CPU/Memory restrictions or even run it in a procd jail to limit the host resource access
|
||||
|
||||
Cons:
|
||||
- It requires to execute a `bbfdmd` instance each module, so takes bit more resouces
|
||||
- Resolving of complex datamodel paths, like datamodel references of external modules (ex: `bridgemngr` refers to wifi layer from `wifidmd`) cost more on IPC
|
||||
|
||||
When not to use micro-service:
|
||||
- Its suggested not to use datamodel micro-service specially in the devices which are having very less resources(in terms of memory/cpu), or
|
||||
- On devices which have high IPC cost over ubus
|
||||
|
||||
### How to Use micro-service
|
||||
|
||||
Manually configuring and starting a micro-services requires multiple steps, but its made simple by using `bbfdm.mk` compile time helper utility.
|
||||
To launch a datamodel definition as a datamodel micro-service, user need to install the datamodel definition/plugin using
|
||||
|
||||
`BBFDM_INSTALL_MS_DM` API from `bbfdm.mk`
|
||||
|
||||
|
||||
```bash
|
||||
bulkdata/Makefile: $(BBFDM_INSTALL_MS_DM) $(PKG_BUILD_DIR)/bbf_plugin/bulkdata.json $(1) $(PKG_NAME)
|
||||
ddnsmngr/Makefile: $(BBFDM_INSTALL_MS_DM) $(PKG_BUILD_DIR)/src/libddnsmngr.so $(1) $(PKG_NAME)
|
||||
```
|
||||
|
||||
Please check [bbfdm.mk](https://dev.iopsys.eu/feed/iopsys/-/blob/devel/bbfdm/bbfdm.mk) documentation for more details.
|
||||
|
||||
In Devices, datamodel micro-services started and managed by `/etc/init.d/bbfdm.services` init script, which reads the global micro-service related configuration from `bbfdm` uci file, and reads module specific configuration for each module's input file present in `/etc/bbfdm/micro_services/` directory.
|
||||
|
||||
|
||||
> In plugins approach, the 3rd party datamodel gets attached to main bbfdm process, which further increases nodes in main datamodel tree, which overtime become a huge tree, results in slower turn around service time for the APIs. Also, any unhandled fault in plugin result in segfault in main bbfdmd process, which takes down the whole tree along with plugin, resulting in complete failure from cwmp and USP, with micro-services plugins runs as an individual processes, so impact of fault limited to its own micro-service only.
|
||||
|
||||
Micro-service approach, disintegrate the plugins further and run them as individual daemons with the help of "bbfdmd" "-m" command line options.
|
||||
|
||||
## Datamodel debugging tools
|
||||
|
||||
To debug the datamodel objects/parameters, `bbfdmd` provides a command line interface, which can
|
||||
|
||||
- Work directly on plugins, or
|
||||
- Gets the data from an ubus object
|
||||
|
||||
and then show it on the CLI. This command line tool is part of `bbfdmd` binary itself and can be accessed with command line argument option '-c' along with binary.
|
||||
To debug the datamodel objects/parameters, `bbfdmd` provides a command line interface, which can gets the data from an ubus object and then show it on the CLI. This command line tool is part of `bbfdmd` binary itself and can be accessed with command line argument option '-c' along with binary.
|
||||
|
||||
```bash
|
||||
# bbfdmd -h
|
||||
Usage: bbfdmd [options]
|
||||
|
||||
options:
|
||||
-s <socket path> ubus socket
|
||||
-m <json path> json input configuration for micro services
|
||||
-c <command input> Run cli command
|
||||
-h Displays this help
|
||||
-l <loglevel> log verbosity value as per standard syslog
|
||||
-h Displays this help
|
||||
|
||||
#
|
||||
```
|
||||
|
|
@ -363,29 +108,6 @@ BBF_DEBUG(MESSAGE, ...)
|
|||
Above API logs gets logged using syslog, based on log_level defined in `bbfdm`.
|
||||
To configure the log_level use `bbfdm.bbfdmd.loglevel` uci option.
|
||||
|
||||
To configure the log_level in micro-service, update the `loglevel` module json file,
|
||||
|
||||
```json
|
||||
# cat /etc/bbfdm/micro_services/sshmngr.json
|
||||
{
|
||||
"cli": {
|
||||
"config": {
|
||||
"proto": "usp" // usp, cwmp
|
||||
},
|
||||
"input": {
|
||||
"type": "UBUS", // JSON, UBUS, DotSO, UNIX
|
||||
"name": "bbfdm"
|
||||
},
|
||||
"output": {
|
||||
"type": "CLI" // CLI, JSON
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
and then restart the bbfdm.services
|
||||
|
||||
## How to extend datamodel
|
||||
|
||||
Although `bbfdm/iowrt` provides datamodels for major services, but still for deployment user might need to add some vendor extensions or needs to add the missing datamodel support. To do the same `bbfdm` or more precisely `libbfdm-api` provides the infrastructure to easily define a new datamodel tree.
|
||||
|
|
|
|||
163
docs/guide/dm-service.md
Normal file
163
docs/guide/dm-service.md
Normal file
|
|
@ -0,0 +1,163 @@
|
|||
# Data Model Micro-Service (dm-service
|
||||
|
||||
`dm-service` daemon is designed to expose a specific module or sub-tree of a data model as datamodel micro-service.
|
||||
|
||||
> Note: The command outputs shown in this document are examples and may vary depending on your device and configuration.
|
||||
|
||||
## Concepts and Workflow
|
||||
|
||||
`dm-service` daemon gets started by `/etc/init.d/bbfdm.services` service. This init script reads input from `bbfdm` UCI file, particularly from `micro_services` section. It then parses each micro-service configuration file located in `/etc/bbfdm/micro_services/`. Each micro-service’s configuration, written in JSON, is used to start the service using the APIs provided by `libbbfdm-ubus` and `libbbfdm-api` libraries.
|
||||
|
||||
`dm-service` daemon use `libbbfdm-api` library to traverse the datamodel tree defined by DotSo or JSON plugin located in `/usr/share/bbfdm/micro_services/$micro-service-name{.so,.json}` and its plugins defined in `/usr/share/bbfdm/micro_services/$micro-service-name/`.
|
||||
|
||||
`dm-service` daemon use `libbbfdm-ubus` library to expose datamodel over UBUS.
|
||||
|
||||
Datamodel micro-service is nothing but another `bbfdmd` instance running with smaller data sub-set, and can be identified at runtime by running a `ps` command
|
||||
|
||||
```bash
|
||||
# ps|grep dm-service
|
||||
12163 root 7664 S {dm_bulkdata} /usr/sbin/dm-service -m bulkdata -l 3
|
||||
12164 root 7620 S {dm_ddnsmngr} /usr/sbin/dm-service -m ddnsmngr -l 3
|
||||
12165 root 7736 S {dm_dhcpmngr} /usr/sbin/dm-service -m dhcpmngr -l 3
|
||||
12166 root 7760 S {dm_dnsmngr} /usr/sbin/dm-service -m dnsmngr -l 3
|
||||
12167 root 7800 S {dm_ethmngr} /usr/sbin/dm-service -m ethmngr -l 3
|
||||
```
|
||||
|
||||
Each `dm-service` instance must be started with -m input to define its service name and run its module(sub-tree) datamodel. These micro-services exposed their own ubus objects
|
||||
|
||||
```bash
|
||||
# ubus list bbfdm.*
|
||||
bbfdm.Bridging
|
||||
bbfdm.BulkData
|
||||
bbfdm.DHCPv4_DHCPv6
|
||||
bbfdm.DNS
|
||||
```
|
||||
|
||||
When a datamodel started as micro-service, it looks/waits for `bbfdm` UBUS object(added by main `bbfdmd` process), once its available micro-service registers with the main `bbfdmd` process by calling the 'service' method
|
||||
|
||||
```bash
|
||||
# ubus -v list bbfdm
|
||||
'bbfdm' @e970413c
|
||||
"service":{"cmd":"String","name":"String","parent_dm":"String","objects":"Array"}
|
||||
```
|
||||
|
||||
## Input and output Schema(s)
|
||||
|
||||
`dm-service` is used to expose sub-tree using different ubus object, UBUS guide and schema for datamodel micro-services
|
||||
|
||||
- [datamodel micro-service](../api/ubus/bbfdm_micro_service.md)
|
||||
- [datamodel micro-service schema](../api/ubus/bbfdm_micro_service.json)
|
||||
|
||||
Apart from these, datamodel micro-services can also be configured by updating their input files directly at runtime, micro-service input files present in `/etc/bbfdm/micro_services/` in CPE.
|
||||
|
||||
A typical micro-service input file looks like below:
|
||||
|
||||
```json
|
||||
{
|
||||
"daemon": {
|
||||
"enable": "1",
|
||||
"service_name": "netmngr",
|
||||
"config": {
|
||||
"loglevel": "3"
|
||||
},
|
||||
"output": {
|
||||
"name": "Network"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Ubus methods
|
||||
|
||||
Following are the UBUS methods exposed by `dm-service` process:
|
||||
|
||||
```bash
|
||||
# ubus -v list bbfdm.Network
|
||||
'bbfdm.Network' @9dc36737
|
||||
"get":{"path":"String","paths":"Array","maxdepth":"Integer","optional":"Table"}
|
||||
"schema":{"path":"String","paths":"Array","first_level":"Boolean","optional":"Table"}
|
||||
"instances":{"path":"String","paths":"Array","first_level":"Boolean","optional":"Table"}
|
||||
"set":{"path":"String","value":"String","datatype":"String","obj_path":"Table","optional":"Table"}
|
||||
"operate":{"command":"String","command_key":"String","input":"Table","optional":"Table"}
|
||||
"add":{"path":"String","obj_path":"Table","optional":"Table"}
|
||||
"del":{"path":"String","paths":"Array","optional":"Table"}
|
||||
```
|
||||
|
||||
> Note1: `optional` table are present in all methods and it supports below options:
|
||||
|
||||
```console
|
||||
"optional":{"proto":"String", "format":"String"}
|
||||
```
|
||||
|
||||
- `proto` in each method specify the data-model prototype('cwmp', 'usp') to use, if not provided default data-model will be used.
|
||||
|
||||
- `format` could be 'raw' or 'pretty', to specify the format to use as output, if not provided 'pretty' format will be used.
|
||||
|
||||
> Note2: `first_level` true means only get next level objects and false means get all objects recursively
|
||||
|
||||
> Note3: `maxdepth` is measured on max number of .(Dot) present in object name
|
||||
|
||||
> Note4: Check ubus schema document for more details
|
||||
|
||||
|
||||
## Pros and Cons of Data Model Micro-Service
|
||||
|
||||
Data model micro-service is nothing but running a partial datamodel sub-set with another instance of `bbfdmd` binary.
|
||||
|
||||
Benefit:
|
||||
- Instead of having a huge datamodel tree, it split the tree based on modules, which reduce the cost of operation on the tree
|
||||
- Resolves one point failure, if any datamodel micro-service not working correctly it only affects that specific module, not the rest of the tree
|
||||
- Moving datamodel code to module repo, brings the service layer and datamodel layer closer, which lowers the possibilities of wrong mapping
|
||||
- Possible to use lightweight `dm-service` inside containers with json plugin for Container service management
|
||||
- Possible to impose CPU/Memory restrictions or even run it in a procd jail to limit the host resource access
|
||||
|
||||
Cons:
|
||||
- It requires to execute a `dm-service` instance each module, so takes bit more resouces
|
||||
- Resolving of complex datamodel paths, like datamodel references of external modules (ex: `bridgemngr` refers to wifi layer from `wifidmd`) cost more on IPC
|
||||
|
||||
When not to use micro-service:
|
||||
- Its suggested not to use datamodel micro-service specially in the devices which are having very less resources(in terms of memory/cpu), or
|
||||
- On devices which have high IPC cost over ubus
|
||||
|
||||
### How to Use micro-service
|
||||
|
||||
Manually configuring and starting a micro-services requires multiple steps, but its made simple by using `bbfdm.mk` compile time helper utility.
|
||||
To launch a datamodel definition as a datamodel micro-service, user need to install the datamodel definition/plugin using
|
||||
|
||||
`BBFDM_INSTALL_MS_DM` API from `bbfdm.mk`
|
||||
|
||||
|
||||
```bash
|
||||
bulkdata/Makefile: $(BBFDM_INSTALL_MS_DM) $(PKG_BUILD_DIR)/bbf_plugin/bulkdata.json $(1) $(PKG_NAME)
|
||||
ddnsmngr/Makefile: $(BBFDM_INSTALL_MS_DM) $(PKG_BUILD_DIR)/src/libddnsmngr.so $(1) $(PKG_NAME)
|
||||
```
|
||||
|
||||
Please check [bbfdm.mk](https://dev.iopsys.eu/feed/iopsys/-/blob/devel/bbfdm/bbfdm.mk) documentation for more details.
|
||||
|
||||
In Devices, datamodel micro-services started and managed by `/etc/init.d/bbfdm.services` init script, which reads the global micro-service related configuration from `bbfdm` uci file, and reads module specific configuration for each module's input file present in `/etc/bbfdm/micro_services/` directory.
|
||||
|
||||
> In plugins approach, the 3rd party datamodel gets attached to main bbfdm process, which further increases nodes in main datamodel tree, which overtime become a huge tree, results in slower turn around service time for the APIs. Also, any unhandled fault in plugin result in segfault in main bbfdmd process, which takes down the whole tree along with plugin, resulting in complete failure from cwmp and USP, with micro-services plugins runs as an individual processes, so impact of fault limited to its own micro-service only.
|
||||
|
||||
Micro-service approach, disintegrate the plugins further and run them as individual daemons with the help of "bbfdmd" "-m" command line options.
|
||||
|
||||
## Datamodel debugging tools
|
||||
|
||||
To configure the log_level in micro-service, update the `loglevel` module json file,
|
||||
|
||||
```json
|
||||
# cat /etc/bbfdm/micro_services/netmngr.json
|
||||
{
|
||||
"daemon": {
|
||||
"enable": "1",
|
||||
"service_name": "netmngr",
|
||||
"config": {
|
||||
"loglevel": "7"
|
||||
},
|
||||
"output": {
|
||||
"name": "Network"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
and then restart the bbfdm.services
|
||||
230
docs/guide/libbbfdm-ubus.md
Normal file
230
docs/guide/libbbfdm-ubus.md
Normal file
|
|
@ -0,0 +1,230 @@
|
|||
# LIBBBFDM Ubus (libbbfdm-ubus)
|
||||
|
||||
`libbbfdm-ubus` is a library that provides APIs to expose datamodel over ubus. It is used to manage various data models in different micro-services and higher-level applications.
|
||||
|
||||
## Use Cases
|
||||
|
||||
`libbbfdm-ubus` library can be used by:
|
||||
|
||||
- `bbfdmd`: to expose the core data model
|
||||
- `dm-service`: to expose a sub-data model as micro-service, example: [Netmngr](https://dev.iopsys.eu/network/netmngr/-/blob/devel/src/net_plugin.c)
|
||||
- Higher-level applications: to expose custom data models as microservices within their daemon, example: [Timemngr](https://dev.iopsys.eu/bbf/timemngr/-/blob/devel/src/main.c)
|
||||
|
||||
## libbbfdm-ubus APIs
|
||||
|
||||
The following APIs are provided by `libbbfdm-ubus` to expose data model over ubus:
|
||||
|
||||
### bbfdm_ubus_regiter_init
|
||||
|
||||
This method is used to initialize the bbfdm_context structure object and register ubus data model methods.
|
||||
|
||||
```
|
||||
int bbfdm_ubus_regiter_init(struct bbfdm_context *bbfdm_ctx)
|
||||
|
||||
inputs
|
||||
struct bbfdm_context *bbfdm_ctx
|
||||
pointer to struct bbfdm_context structure to be initialized.
|
||||
|
||||
return
|
||||
int fault
|
||||
returns 0 on success, or an error code if the registration fails.
|
||||
```
|
||||
|
||||
|
||||
### bbfdm_ubus_regiter_free
|
||||
|
||||
This method is used to free the bbfdm_context structure object
|
||||
|
||||
```
|
||||
int bbfdm_ubus_regiter_free(struct bbfdm_context *bbfdm_ctx)
|
||||
|
||||
inputs
|
||||
struct bbfdm_context *bbfdm_ctx
|
||||
pointer to struct bbfdm_context structure to be freed.
|
||||
|
||||
return
|
||||
int fault
|
||||
returns 0 on success, or an error code if freeing fails.
|
||||
```
|
||||
|
||||
|
||||
### bbfdm_ubus_set_service_name
|
||||
|
||||
This method is used the service name for the daemon running as a microservice
|
||||
|
||||
```
|
||||
void bbfdm_ubus_set_service_name(struct bbfdm_context *bbfdm_ctx, const char *srv_name)
|
||||
|
||||
input
|
||||
struct bbfdm_context *bbfdm_ctx
|
||||
pointer to struct bbfdm_context structure
|
||||
|
||||
const char *srv_name
|
||||
pointer to service name to set
|
||||
|
||||
return
|
||||
None
|
||||
```
|
||||
|
||||
|
||||
### bbfdm_ubus_set_log_level
|
||||
|
||||
This method is used to set the log level according to the standard syslog levels
|
||||
|
||||
```
|
||||
void bbfdm_ubus_set_log_level(int log_level)
|
||||
|
||||
input
|
||||
int log_level
|
||||
desired log level to set
|
||||
|
||||
return
|
||||
None
|
||||
```
|
||||
|
||||
|
||||
### bbfdm_ubus_load_data_model
|
||||
|
||||
This method is used to load an internal data model, allowing you to use an internal model instead of external plugins (e.g., DotSo or JSON).
|
||||
|
||||
```
|
||||
void bbfdm_ubus_load_data_model(DM_MAP_OBJ *DynamicObj)
|
||||
|
||||
input
|
||||
DM_MAP_OBJ *DynamicObj
|
||||
pointer to internal data model
|
||||
|
||||
return
|
||||
None
|
||||
```
|
||||
|
||||
## libbbfdm-ubus methods
|
||||
|
||||
Following are the ubus methods exposed by `libbbfdm-ubus` when registering a new module:
|
||||
|
||||
```bash
|
||||
# ubus -v list bbfdm
|
||||
'bbfdm' @9e9928ef
|
||||
"get":{"path":"String","paths":"Array","maxdepth":"Integer","optional":"Table"}
|
||||
"schema":{"path":"String","paths":"Array","first_level":"Boolean","optional":"Table"}
|
||||
"instances":{"path":"String","paths":"Array","first_level":"Boolean","optional":"Table"}
|
||||
"set":{"path":"String","value":"String","obj_path":"Table","optional":"Table"}
|
||||
"operate":{"command":"String","command_key":"String","input":"Table","optional":"Table"}
|
||||
"add":{"path":"String","obj_path":"Table","optional":"Table"}
|
||||
"del":{"path":"String","paths":"Array","optional":"Table"}
|
||||
"service":{"cmd":"String","name":"String","parent_dm":"String","objects":"Array"}
|
||||
"notify_event":{"name":"String","input":"Array"}
|
||||
```
|
||||
|
||||
## libbbfdm-ubus example(s)
|
||||
|
||||
1. The requested value is correct as per TR181 standard, but there is a limitation in the device.
|
||||
|
||||
```console
|
||||
root@iopsys:~# ubus call bbfdm set '{"path":"Device.Firewall.Config", "value":"High"}'
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"path": "Device.Firewall.Config",
|
||||
"fault": 9007,
|
||||
"fault_msg": "The current Firewall implementation supports only 'Advanced' config."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
2. The requested value is outside the allowed range.
|
||||
|
||||
```console
|
||||
root@iopsys:~# ubus call bbfdm set '{"path":"Device.Firewall.Chain.1.Rule.9.DestPort", "value":"123456"}'
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"path": "Device.Firewall.Chain.1.Rule.9.DestPort",
|
||||
"fault": 9007,
|
||||
"fault_msg": "'123456' value is not within range (min: '-1' max: '65535')"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
3. Some arguments should be defined to perform the requested operation.
|
||||
|
||||
```console
|
||||
root@iopsys:~# ubus call bbfdm operate '{"command":"Device.IP.Diagnostics.IPPing()", "command_key":"ipping_test", "input":{}}'
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"path": "Device.IP.Diagnostics.IPPing()",
|
||||
"data": "ipping_test",
|
||||
"fault": 7004,
|
||||
"fault_msg": "IPPing: 'Host' input should be defined"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
4. The path parameter value must start with 'Device.'. The command below doesn't have Device before path "Users.User."
|
||||
|
||||
```console
|
||||
root@iopsys:~# ubus call bbfdm get '{"path":"Users.User.", "optional": {"format":"raw", "proto":"usp"}}'
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"path": "Users.User.",
|
||||
"fault": 7026,
|
||||
"fault_msg": "Path is not present in the data model schema"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
These fault messages defined in datamodel handlers, users can add such fault message using `bbfdm_set_fault_message` libbbfdm-api's API, if no specific fault message defined for particular obj/param, datamodel returns standard error messages that are defined in CWMP and USP protocols as the fault message value.
|
||||
|
||||
### Fault handling
|
||||
|
||||
To indicate a fault and source of fault, `libbbfdm-ubus` provides `fault` along with `fault_msg` in the response in case of faults, which then handled by higher layer applications (i.e icwmp, obuspa).
|
||||
|
||||
This provides a clear inside on the root cause of the fault, and based on `fault_msg` it's easily to understand what the issue is and how to fix it and find out the limitations(if there are any on the device).
|
||||
|
||||
##### Errors Codes
|
||||
|
||||
| Error Code | Meaning |
|
||||
|------------|--------------------------------------------------------------|
|
||||
| 7003 | Message failed due to an internal error. |
|
||||
| 7004 | Message failed due to invalid values in the request elements and/or failure to update one or more parameters during Add or Set requests. |
|
||||
| 7005 | Message failed due to memory or processing limitations. |
|
||||
| 7008 | Requested path was invalid or a reference was invalid. |
|
||||
| 7010 | Requested Path Name associated with this ParamError did not match any instantiated parameters. |
|
||||
| 7011 | Unable to convert string value to correct data type. |
|
||||
| 7012 | Out of range or invalid enumeration. |
|
||||
| 7022 | Command failed to operate. |
|
||||
| 7026 | Path is not present in the data model schema. |
|
||||
|
||||
|
||||
## Parallel calls over Ubus
|
||||
|
||||
Some datamodel operations takes less time to execute compared to other, like
|
||||
- Get on sub-set of datamodel or an individual datamodel parameter takes very less, where as
|
||||
- Get on complete Device. and Async operate commands takes much longer
|
||||
|
||||
executing/serializing operations simplifies the code from developer perspective, but its not suitable for deployments. To make it suitable `bbfdmd` support parallel calls.
|
||||
|
||||
- All datamodel `operate` commands are running in parallel
|
||||
|
||||
example(s):
|
||||
|
||||
```console
|
||||
root@iopsys:~# time ubus call bbfdm get '{"path":"Device."}' >/dev/null &
|
||||
root@iopsys:~# time ubus call bbfdm get '{"path":"Device.Users."}' >/dev/null
|
||||
real 0m 0.07s
|
||||
user 0m 0.00s
|
||||
sys 0m 0.00s
|
||||
root@iopsys:~#
|
||||
real 0m 1.86s
|
||||
user 0m 0.05s
|
||||
sys 0m 0.00s
|
||||
|
||||
[1]+ Done time ubus call bbfdm get "{\"path\":\"Device.\"}" >/dev/null
|
||||
root@iopsys:~#
|
||||
```
|
||||
|
|
@ -11,7 +11,7 @@ The Activate() command has as arguments the TimeWindow object which is used to a
|
|||
Below is an example of an 'Activate()' command call with three TimeWindow instances. As a result, three jobs are created according to the defined TimeWindow.{i}.Start:
|
||||
|
||||
```bash
|
||||
root@iopsys-44d43771aff0:~# ubus call usp operate '{"path":"Device.DeviceInfo.FirmwareImage.2.", "action":"Activate()", "input":{"TimeWindow.1
|
||||
root@iopsys-44d43771aff0:~# ubus call bbfdm operate '{"path":"Device.DeviceInfo.FirmwareImage.2.", "action":"Activate()", "input":{"TimeWindow.1
|
||||
.Start":"1800", "TimeWindow.1.End":"3600", "TimeWindow.1.Mode":"WhenIdle", "TimeWindow.2.Start":"5400", "TimeWindow.2.End":"9000", "TimeWindow
|
||||
.2.Mode":"WhenIdle", "TimeWindow.3.Start":"86400", "TimeWindow.3.End":"172800", "TimeWindow.3.Mode":"Immediately"}}'
|
||||
{
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue