mirror/bbfdm
1
0
Fork 0
mirror of https://dev.iopsys.eu/bbf/bbfdm.git synced 2025-12-10 07:44:39 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Updated document for micro-service

Browse source
This commit is contained in:
Vivek Kumar Dutta 2023-07-18 20:28:56 +05:30
parent 083b66f7b5
commit 5bab19344c
3 changed files with 199 additions and 74 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

35
README.md
View file

@ -1,21 +1,19 @@
# BroadBand Forum Data Models (BBFDM)
`bbfdm` is an implementation developed by iopsys that conforms to Broadband Forum Data Models and which includes a list objects, parameters, events and operates used for CPE management through remote control protocols such as [TR-069/CWMP](https://cwmp-data-models.broadband-forum.org/) or [TR-369/USP](https://usp.technology/).
`bbfdm` is a datamodel backend for Higher layer management protocols like [TR-069/CWMP](https://cwmp-data-models.broadband-forum.org/) or [TR-369/USP](https://usp.technology/). It is designed in a hardware agnostic way and provides the available datamodel parameters over ubus on the northbound interface and creates the datamodel mapping based on uci and ubus on southbound interface.
This implementation comprises of the three main components:
`bbfdm` has three main components:
| Component | Description |
| ---------- | ------------------------------------------------- |
| libbbfdm-api | It is a library that provides many APIs used to interact with UCI configurations, Ubus objects, JSON schema, CLI commands and memory management. It also provides a mechanism to add new objects, parameters, events and operates or extend the existing DM tree using json plugin or shared library plugin. |
| libbbfdm | It's a libarry that provides the different data models supported by iopsys |
| bbfdmd | It's a deamon which used to expose data model objects over ubus |
| bbfdmd | A daemon to expose data model objects over ubus in pretty and raw format |
| libbbfdm-api | A shared library which provides API to build and parse datamodel tree, it also provides API to create datamodel extensions using shared DotSo plugin, or with JSON Plugin. |
| libbbfdm | A datamodel tree/library build with libbbfdm-api, it includes core TR181 and related datamodel |
## Design of bbfdm
`bbfdm` package is structred as follow:
## Directory Structure
`bbfdm` package is structured as follow:
```bash
├── bbfdmd
@ -24,7 +22,6 @@ This implementation comprises of the three main components:
├── libbbfdm
│   ├── dmtree
│   │   ├── json
│   │   ├── tr104
│   │   ├── tr143
│   │   ├── tr181
│   │   ├── tr471
@ -34,7 +31,7 @@ This implementation comprises of the three main components:
```
- `bbfdmd` folder which contains the source code of bbfdm deamon.
More explanation on how this daemon works and all supported methods are presented in this file [BBFDMD](./bbfdmd/README.md)
More explanation on how this daemon works and all supported methods are presented in this file [BBFDMD](./docs/arch/bbfdmd.md)
- `libbbfdm` folder which contains the different data models supported by iopsys
@ -42,8 +39,6 @@ More explanation on how this daemon works and all supported methods are presente
- `tr181` folder : TR-181 Data Model files
- `tr104` folder : Voice Services Data Model files
- `tr143` folder : Diagnostics Data Model files
- `tr471` folder : IPLayerCapacityMetrics Diagnostics Data Model files
@ -63,20 +58,23 @@ All supported tools are presented in this file[BBFDM Tools](./docs/guide/tools.m
- `docs` folder which contains all documentation files.
## Important Topics
* [Design for firmware activation](./docs/guide/activate_firmware.md)
* [TR181 Firewall datamodel mappings](./docs/guide/firewall.md)
## Design
* [BBFDMD Design](./docs/arch/bbfdmd.md)
* [Datamodel extension using JSON plugin](./docs/guide/json_plugin_v1.md)
* [Datamodel Plugins and Microservice](./docs/guide/datamodel_as_microservice.md)
* [BBFDM Tools](./docs/guide/tools.md)
## Important Topics
* [Add support of a new Object/Parameter](./docs/guide/obj_param_extension.md)
* [How to add new vendor](./docs/guide/vendor.md)
* [Dynamic Object/Parameter/Operate/Event](./docs/guide/dynamic_dm.md)
* [BBFDM Tools](./docs/guide/tools.md)
* [Design for firmware activation](./docs/guide/activate_firmware.md)
* [TR181 Firewall datamodel mappings](./docs/guide/firewall.md)
* [Wireless Configuration handling](./docs/guide/wireless_easymesh.md)
* [Explain the different Network Deployment Scenarios](./docs/guide/network_depoyment_scenarios.md)
* [How to Configure MACVLAN](./docs/guide/macvlan_interface.md)
* [Explain Policy Based Routing Management](./docs/guide/policy_based_routing.md)
## External dependencies for datamodel objects
| Datamodel | Package | Link |
@ -92,3 +90,4 @@ All supported tools are presented in this file[BBFDM Tools](./docs/guide/tools.m
| STUN parameters | stunc | https://dev.iopsys.eu/bbf/stunc.git |
| Device.XPON. | ponmngr | https://dev.iopsys.eu/hal/ponmngr.git |
| Device.UPNP. | ssdpd | https://github.com/miniupnp/miniupnp.git |
| Device.Services.VoiceService. | tr104 | https://dev.iopsys.eu/voice/tr104.git |

112
bbfdmd/README.md → docs/arch/bbfdmd.md
View file

@ -29,7 +29,7 @@ In the above uci, loglevel can have below value:
| 3 | Log everything except debug |
| 4 | Everything will be logged |
For more info on the `bbfdmd` UCI configuration visit [uci documentation](../docs/api/uci/bbfdm.md) OR [raw uci schema](../schemas/uci/bbfdm.json)
For more info on the `bbfdmd` UCI configuration visit [uci documentation](../docs/api/uci/bbfdm.md) OR [raw uci schema](../../schemas/uci/bbfdm.json)
## Concepts and Workflow
@ -39,20 +39,67 @@ When a ubus method is called it first fills `bbfdm_data_t` structure with the ne
`bbfdmd` uses `bbf_entry_method` API from `libbbfdm-api` and `tEntryRoot`, `tVendorExtension`, `tVendorExtensionOverwrite` and `tVendorExtensionExclude` global shared arrays from `libbbfdm` to get the device tree schema and its values.
In short, it covers/supports all methods introduced in `TR-069` and `TR-369` by using the `bbf_entry_method` API from `libbbfdm-api` with the differents methods and the existing data-model available with `libbbfdm`.
In short, it covers/supports all methods introduced in `TR-069` and `TR-369` by using the `bbf_entry_method` API from `libbbfdm-api` with the different methods and the existing data-model available with `libbbfdm`.
## BBFDMD Command Line Arguments
## Debugging tools
With the advancement in the datamodel tree, it is sometime required to do some debugging at the source, to simplify that `bbfdmd` offer a command line tool, which can
`bbfdmd` supports two modes, a daemon mode (seen above) and a command (or CLI) mode, which supports interactively querying the data model and setting values in the configuration via the data model.
- Work directly on plugins, or
- Gets the data from an ubus object
Actually, the CLI mode is an utility to simplify the interaction with data model that gives to customer the visibility to expose any kind of data model (which can be a `DotSo` plugin, `JSON` plugin, `UBUS` command or `UNIX` socket) with a specific format (`CLI` or `JSON`).
and then show it on the CLI. This command line tool is part of `bbfdmd` binary itself and can be accessed with command line argument option '-c' along with binary.
The CLI mode is specified with the `-c` option and can be run using `cwmp` or `usp` protocol.
```bash
# bbfdmd -h
Usage: bbfdmd [options]
All of the above configurations should be done by json file which can be located anywhere, just don't forget to pass the path with `-I` option.
options:
-s <socket path> ubus socket
-m <json path> json input configuration for micro services
-c <command input> Run cli command
-h Displays this help
```console
root@iopsys:~# bbfdmd -I /tmp/test.json -c help
#
```
If no command line option provided along with `bbfdmd` command then it starts in daemon mode and get the default configuration from `/etc/bbfdm/input.json`
```bash
# cat /etc/bbfdm/input.json
{
"daemon": {
"config": {
"loglevel": "1",
"refresh_time": "10",
"transaction_timeout": "10"
},
"input": {
"type": "DotSo",
"name": "/lib/libbbfdm.so"
},
"output": {
"type": "UBUS",
"name": "bbfdm"
}
},
"cli": {
"config": {
"proto": "both",
"instance_mode": 0
},
"input": {
"type": "UBUS",
"name": "bbfdm"
},
"output": {
"type": "CLI"
}
}
}
```
In command (or CLI) mode, it supports interactively querying the data model and setting values in the configuration via the data model.
```bash
# bbfdmd -c help
Valid commands:
help
get [path-expr]
@ -61,10 +108,10 @@ Valid commands:
del [path-expr]
instances [path-expr]
schema [path-expr]
#
```
Below is an example of json file:
Below is another example of json input file:
```json
{
@ -83,38 +130,10 @@ Below is an example of json file:
}
}
```
> NOTE1: `bbfdmd` CLI mode is an experimentation feature and it can be updated later.
> NOTE2: If `-I` option is not passed when starting `bbfdmd`, so configuration options will be loaded from the default [INPUT.JSON](../../json/input.json) located in '/etc/bbfdm/input.json'.
* To see a list of arguments supported by `bbfdmd` use:
```console
root@iopsys:~# bbfdmd -h
Usage: bbfdmd [options]
options:
-s <socket path> ubus socket
-I <json path> json input configuration
-c <command input> Run cli command
-h Displays this help
```
* To see a list of commands supported by `bbfdmd` in CLI mode use:
```console
root@iopsys:~# bbfdmd -c help
Valid commands:
help
get [path-expr]
set [path-expr] [value]
add [object]
del [path-expr]
instances [path-expr]
schema [path-expr]
```
> NOTE2: If `-m` option is not passed when starting `bbfdmd`, so configuration options will be loaded from the default [INPUT.JSON](../../json/input.json) located in '/etc/bbfdm/input.json'.
* To see the currently implemented data model use:
@ -127,19 +146,12 @@ $ bbfdmd -c schema Device.
```console
root@iopsys:~# bbfdmd -c get Device.Time.
Device.Time.Enable => 0
Device.Time.Status => Disabled
Device.Time.NTPServer1 => ntp1.sth.netnod.se
Device.Time.NTPServer2 => ntp1.gbg.netnod.se
Device.Time.NTPServer3 =>
Device.Time.NTPServer4 =>
Device.Time.NTPServer5 =>
Device.Time.CurrentLocalTime => 2023-04-22T13:45:01+00:00
Device.Time.LocalTimeZone => CET-1CEST,M3.5.0,M10.5.0/3
root@iopsys:~#
root@iopsys:~# bbfdmd -c get Device.WiFi.SSID.1.SSID
Device.WiFi.SSID.1.SSID => test-5g
```
> Note: The "parameter" may contain wildcard intance and partial paths.
> Note: Wildcard (*) is valid placeholder for multi-instance object instance and partial paths are also allowed.
* To set the value of a data model parameter use:

114
docs/guide/datamodel_as_microservice.md Normal file
View file

@ -0,0 +1,114 @@
# Datamodel additions (DotSO Plugin/ JSON Plugin/ Micro-Service)
It is often required by customers to have/implement some datamodel parameters which is not available in bbfdm, `bbfdm` provides four ways to extend the existing datamodel parameters.
- Using [JSON Plugin](./json_plugin_v1.md)
- Using [DotSO Plugin](./bbfdm_dm_integration.md)
- With bbfdm micro-service
- [Individual datamodel daemon](https://dev.iopsys.eu/voice/tr104)
Out of all the supported method, each have its own pro and cons:
## JSON Plugins
Pro:
- Easy to add (compilation not required)
- Least maintenance (Change in libbbfdm-api has minimal impact)
Con:
- Can only support easy one to one mappings with uci and ubus
- In-tree datamodel references not supported
- Invalid plugin syntax might cause segfault on main bbfdmd service, result in complete cwmp/usp service down
- Make the main tree bigger, which result into slow service time
## DotSO Plugin
Pro:
- Similar to adding in the core
- Support complex mapping and data sharing between nodes
- All core operations supported
Con:
- Moderate maintenance (Change in libbbfdm-api requires adaptation/alignment)
- Any segfault on plugin might teardown main bbfdmd service, result in complete cwmp/usp service down
- Make the main tree bigger, which result into slow service time
## BBFDM micro-service
Pro:
- Creates a sub datamodel tree and registers to main datamodel tree
- Can be used with existing DotSO and JSON Plugins
- Faults in plugins only limited to micro-service, main tree won't gets affected
Con:
- Currently references to out side tree nodes not supported
- Maintenance cost depends on JSON or DotSO plugin selection
## Individual daemon to run micro-service
It has same pro and cons as of running the plugin with micro-service, with a added benefit of low maintenance as all the dependency embedded in single binary.
## Workflow of Individual daemon and micro-service
daemon started by the init script, it adds its own ubus objects based on input and wait for parent/master ubus object, once master ubus object available, it registers self with the main tree.
`icwmp` and `obuspa` gets the datamodel from main tree which is provided by `bbfdm` daemon, so no change required in either cwmp and obuspa.
## Summary
In plugins approach, the 3rd party datamodel gets attached to main bbfdm process, which further increases nodes in main datamodel tree, which overtime become a huge tree, results in slower turn around service time for the APIs.
Also with plugins, since its part of main process, any unhandled fault result in segfault in main bbfdmd process, which takes down the whole tree along with it, resulting in complete failure from cwmp and USP, with micro-services plugins runs as an individual processes, so impact of fault limited to its own process only.
Micro-service approach, disintegrate the plugins further and run them as individual daemons with the help of "bbfdmd" "-m" command line options.
## How to switch to micro-service model
It's a three step process, if DotSO or JSON plugin already present, if not refer to the plugins document.
1. Install the DotSO/JSON plugin to non-bbf plugin location
2. Create an input.json file, like below
```json
{
"daemon": {
"input": {
"type": "JSON", // JSON or DotSO
"name": "/etc/bulkdata/bulkdata.json" // Path of the plugin
},
"output": {
"type": "UBUS",
"name": "bbfdm.bulkdata", // name of the ubus object to register for micro-service
"parent_dm": "Device.", // Parent tree to attach the plugin
"object": "BulkData", // Name of the object
"root_obj": "bbfdm" // Name of the root tree ubus object which is bbfdm
}
}
}
```
3. Update init script to start the datamodel micro-service, which made simpler with `bbfdm` [init hooks](https://dev.iopsys.eu/feed/iopsys/-/commit/8bdfd3ea51a81941ee9c53a46a66cf6c0f6eb88f)
```bash
. /etc/bbfdm/bbfdm_services.sh
bbfdm_add_service "bbfdm.bulkdata" "/etc/bulkdata/input.json"
```
## When to switch to micro-service model
There are few parameters which can help in answer this query,
1. If a plugin add significantly huge amount of nodes to main tree
2. If the service known to miss-behave and cause faults
3. If a plugin add a service with no/less dependency on other datamodel parameters
## DotSO Plugins examples
- [STUN](https://dev.iopsys.eu/bbf/stunc.git)
- [UDPEcho](https://dev.iopsys.eu/bbf/udpecho.git)
- [SoftwareModule](https://dev.iopsys.eu/bbf/swmodd.git)
- [XMPP](https://dev.iopsys.eu/bbf/xmpp.git)
- [TWAMP](https://dev.iopsys.eu/bbf/twamp-light.git)
- [Usermngr](https://dev.iopsys.eu/bbf/usermngr.git)
- [ManagementServer](https://dev.iopsys.eu/bbf/icwmp.git)
## JSON Plugin example
- [URLFilter](https://dev.iopsys.eu/feed/iopsys/-/blob/devel/urlfilter/files/etc/bbfdm/json/urlfilter.json)
- [XPON](https://dev.iopsys.eu/feed/iopsys/-/blob/devel/ponmngr/files/etc/bbfdm/json/xpon.json)
## Micro service example
- [Bulkdata](https://dev.iopsys.eu/feed/iopsys/-/commit/8bdfd3ea51a81941ee9c53a46a66cf6c0f6eb88f)
- [PeriodicStats](https://dev.iopsys.eu/feed/iopsys/-/commit/66163d394586b953b8f891f91afb1677df29403a)
## Individual daemons
- [TR104](https://dev.iopsys.eu/feed/iopsys/-/commit/7160fadf5607fcc785fd13e41eac402da6164280)