Robot Chef distributed in multiple packages

In previous chapter, Robot Chef was written using lml but in a single package and its plugins are loaded immediately. In this chapter, we will decouple the plugin and the main package using lml. And we will demonstrates the changes needed to plugin them back with the main package.

Demo

Do the following:

$ git clone https://github.com/python-lml/robotchef
$ cd robotchef
$ python setup.py install

The main command line interface module does simply this:

$ robotchef "Portable Battery"
I can cook Portable Battery for robots

Although it does not understand all the cuisines in the world as you see as below:

$ robotchef "Jacket Potato"
I do not know how to cook Jacket Potato

it starts to understand it once you install Chinese cuisine package to complement its knowledge:

$ git clone https://github.com/python-lml/robotchef_britishcuisine
$ cd robotchef_britishcuisine
$ python setup.py install

And then type in the following:

$ robotchef "Fish and Chips"
I can fry Fish and Chips

The more cuisine packages you install, the more dishes it understands. Here is the loading sequence:

Decoupling the plugins with the main package

In order to demonstrate the capabilities of lml, Boost class is singled out and placed into an internal module robotchef.robot_cuisine. Fry and Bake are relocated to robotchef_britishcuisine package, which is separately installable. built-in and standalone-plugin will explain how to glue them up.

After the separation, in order to piece all together, a special function lml.loader.scan_plugins() needs to be called before the plugins are used.

--- /home/docs/checkouts/readthedocs.org/user_builds/lml/checkouts/latest/examples/robotchef_allinone_lml/robotchef_allinone_lml/main.py
+++ /home/docs/checkouts/readthedocs.org/user_builds/lml/checkouts/latest/examples/robotchef/robotchef/main.py
@@ -1,6 +1,16 @@
 import sys
+import logging
+import logging.config
 
-from robotchef_allinone_lml.plugin import CuisineManager, NoChefException
+from lml.loader import scan_plugins_regex
+from robotchef.plugin import CuisineManager, NoChefException
+
+logging.basicConfig(
+    format="%(name)s:%(lineno)d - %(levelname)s - %(message)s",
+    level=logging.DEBUG,
+)
+
+BUILTINS = ["robotchef.robot_cuisine"]
 
 
 def main():
@@ -8,6 +18,11 @@
         sys.exit(-1)
 
     cuisine_manager = CuisineManager()
+    scan_plugins_regex(
+        plugin_name_patterns="robotchef_",
+        pyinstaller_path="robotchef",
+        white_list=BUILTINS,
+    )
 
     food_name = sys.argv[1]
     try:

What’s more, lml.loader.scan_plugins() search through all installed python modules and register plugin modules that has prefix “robotchef_”.

The second parameter of scan_plugins is to inform pyinstaller about the package path if your package is to be packaged up using pyinstaller. white_list lists the built-ins packages.

Once scan_plugins is executed, all ‘cuisine’ plugins in your python path, including the built-in ones will be discovered and will be collected by PluginInfoChain in a dictionary for get_a_plugin() to look up.

Plugin management

As you see in the class relationship diagram, There has not been any changes for CuisineManager which inherits from :class:lml.PluginManager and manages cuisine plugins. Please read the discussion in previous chapter. Let us look at the plugins.

Built-in plugin

Boost plugin has been placed in a submodule, robotchef.robot_cuisine. Let us see how it was done. The magic lies in robot_cuisine module’s __init__.py

from lml.plugin import PluginInfoChain

PluginInfoChain(__name__).add_a_plugin(
    "cuisine", "electrify.Boost", tags=["Portable Battery"]
)

A unnamed instance of lml.plugin.PluginInfoChain registers the meta data internally with CuisineManager. __name__ variable refers to the module name, and in this case it equals ‘robotchef.robot_cuisine’. It is used to form the absolute import path for Boost class.

First parameter cuisine indicates that electrify.Boost is a cuisine plugin. lml will associate it with CuisineManager. It is why CuisineMananger has initialized as ‘cuisine’. The second parameter is used the absolute import path ‘robotchef.robot_cuisine.electricity.Boost’. The third parameter tags are the dictionary keys to look up class Boost.

Here is a warning: to achieve lazy loading as promised by lml, you shall avoid heavy duty loading in __init__.py. this design principle: not to import any un-necessary modules in your plugin module’s __init__.py.

That’s all you need to write a built-in plugin.

Standalone plugin

Before we go to examine the source code of robotchef_britishcuisine, please let me dictate that the standalone plugins shall respect the package prefix, which is set by the main package. In this case, the plugin packages shall start with ‘robotchef_’. Hence for British Cuisine, it is named as ‘robotchef_britishcuisine’.

Now let us have look at the module’s __init__.py, you would find similar the plugin declaration code as in the following. But nothing else.

from lml.plugin import PluginInfoChain

PluginInfoChain(__name__).add_a_plugin(
    "cuisine", "fry.Fry", tags=["Fish and Chips"]
).add_a_plugin("cuisine", "bake.Bake", tags=["Cornish Scone", "Jacket Potato"])

Because we have relocated Fry and Bake in this package, the instance of PluginInfoChain issues two chained call add_a_plugin() but with corresponding parameters.

Note

In your plugin package, you can add as many plugin class as you need. And the tags can be as long as you deem necessary.

Let me wrap up this section. All you will need to do, in order to make a standalone plugin, is to provide a package installer(setup.py and other related package files) for a built-in plugin.

The end

That is all you need to make your main component to start using component based approach to expand its functionalities. Here is the takeaway for you:

1. lml.plugin.PluginManager is just another factory pattern that hides the complexity away.
2. You will need to call lml.loader.scan_plugins() in your __init__.py or where appropriate before your factory class is called.

More standalone plugins

You are left to install robotchef_chinesecuisine and robotchef_cook yourself and explore their functionalities.

How to ask robotchef to forget British cuisine?

The management of standalone plugins are left in the hands of the user. To prevent robotchef from finding British cuisine, you can use pip to uninstall it, like this:

$ pip uninstall robotchef_britishcuisine