NCNN adopts the factory pattern to create the layers of a nueral network. It's also the way the well-known library Caffe takes. It differs from Caffe in the implementation of the registry table. On one hand, the Caffe registry is populated in runtime as the side effect of initializion of global variable (which is a popular way for library initialization). On the other hand, the NCNN registry is determined in compile time. The registry is generated in a brilliant way using CMake instead of a hand-crafted table. NCNN's approach provides several benefits compared to Caffe's approach.
First, it's suitable for building a static library. When building a static library, the linker will strip any unused global variable to minimize the size of the library. This makes sense but it also strips the global variable which need to be inintialized to insert te layer creator into the registry. Tricky linker flags and related instrutions are required to resolve this issue. By creating the registry in compile time, it ensures the completeness of the registry when building static library.
Second, the size of the library can be minimzed. In Caffe's approach, the registry is registered during the initialization of a global variable defined in the layer implementation file. As a result all of the layer implementation file is compiled into the library. In constrast, NCNN selects the implementation files using a macro to compile into the library because the registry is known in compile time.
The infrastructure of factory pattern is defined in the file layer.h and layer.cpp. The layer creator function layer_creator_func and the create method Layer* create_layer() are declared as the following. Note that there is NCNN_STRING flag to enable or disable the usage of string. This flag alters the structure of the registry and the creation method. It is used to further minimize the size of the library. Normally, it is simply defined.
// layer factory function
typedef Layer* (*layer_creator_func)();
#if NCNN_STRING
// get layer type from type name
int layer_to_index(const char* type);
// create layer from type name
Layer* create_layer(const char* type);
#endif // NCNN_STRING
// create layer from layer type
Layer* create_layer(int index);In NCNN, the registry layer_registry consists of a list of entries instead of a map in Caffe. It includes the layer_registry.h which is populated during CMake configuration which will be mentioned in the next section. It contains the information of layer name and its creator.
struct layer_registry_entry
{
#if NCNN_STRING
// layer type name
const char* name;
#endif // NCNN_STRING
// layer factory entry
layer_creator_func creator;
};
static const layer_registry_entry layer_registry[] =
{
#include "layer_registry.h"
};
In file layer.cpp, the implementation of create_layer() method simply traverses the registry to match the layer name and return the assocaited creator.
#if NCNN_STRING
int layer_to_index(const char* type)
{
for (int i=0; i<layer_registry_entry_count; i++)
{
if (strcmp(type, layer_registry[i].name) == 0)
return i;
}
return -1;
}
Layer* create_layer(const char* type)
{
int index = layer_to_index(type);
if (index == -1)
return 0;
return create_layer(index);
}
#endif // NCNN_STRING
Layer* create_layer(int index)
{
if (index < 0 || index >= layer_registry_entry_count)
return 0;
layer_creator_func layer_creator = layer_registry[index].creator;
if (!layer_creator)
return 0;
Layer* layer = layer_creator();
layer->typeindex = index;
return layer;
}The CMake file gathers the source files and builds the ncnn static library. During configuration, it generates the layer_declaration.h, layer_registry.h and layer_type_enum.h which contain the essence part of factory pattern according to user-provided options to turn on or off the compilation of specific layers.
NCNN uses a fancy way to find whether there exists a architecture specific implemention for the layer. For exmaple, given layer ${class} and architecture ${arch}. First, convert ${class} to all lowercase ${name}. Second, it checks if there exists a file in the path ./layer/${arch}/${name}_${arch}.cpp which is the where the architecure specific file locates. On the other hand, the architecure independent implementation file locates at ./layer/${name}.cpp.
macro(ncnn_add_layer class)
string(TOLOWER ${class} name}
......
else()
option(WITH_LAYER_${name} “...” ON)
endif()
......
if(WITH_LAYER_${name})
# Append arch-independent impl file to build
list(APPEND ncnn_SRCS .../layer/${name}.cpp)
# Determine current arch
......
else()
set(arch x86)
endif()
# Look for the optimized impl file for this arch
# The file path should follow the rule
set(LAYER_ARCH_SRC .../layer/${arch}/${name}_${arch}.cpp)
if(EXISTS ${LAYER_ARCH_SRC})
set(WITH_LAYER_${name}_${arch} 1)
list(APPEND ncnn_SRCS ${LAYER_ARCH_SRC})
endif()
.......
endif()
if(WITH_LAYER_${name})
# Append arch-independent to `layer_declaration`
set(layer_declaration “${layer_declaration} ...”)
endif()
if(WITH_LAYER_${name}_${arch})
# Append arch-specific to `layer_declaration`
set(layer_declaration “${layer_declaration} ...”)
endif
......
if(WITH_LAYER_${name})
# Append the creator to `layer_registry`
set(layer_registry “${layer_registry} {\”${class}\”, ${class}_final_layer_creator}\n”)
endif()
# Type enum file
set(layer_type_enum “${layer_type_enum}${class} = ${idx}”)
math(EXPR idx “${idx}+1”)
endmacro()
......
ncnn_add_layer(Convolution)
......
configure_file(layer_declaration.h.in .../layer_declaration.h)
configure_file(layer_registry.h.in .../layer_registry.h)
configure_file(layer_type_enum.h.in .../layer_type_enum.h)
......
add_library(ncnn STATIC ${ncnn_SRCS})There are three template files (more if count gpu support using Vulkan). All of them only contains a placeholder for subsitution with Cmake variables to fullfill the NCNN's factory pattern.
File layer_declaration.h.in
@layer_declaration@
File layer_registry.h.in
@layer_registry@
File layer_type_enum.h.in
@layer_type_enum@
This is generated from layer_declaration.h.in by subsituting @layer_declaration@ with Cmake variable layer_declaration during Cmake configuration. This file creates a new class ${name}_final which inherits from arch-independent and arch-sepcific implementation classes. This new class is then wrapped as the final creator using C macro named ##name_layer_creator().
.........
#include “layer/convolution.h”
#include “layer/x86/convolution_x86.h”
namespace ncnn {
class Convolution_final :
virtual public Convolution,
virtual public Convolution_x86
{
public:
virtual int create_pipeline(const Option& opt)
{ ...; }
virtual int destroy_pipeline(const Option& opt)
{ ...; }
};
DEFINE_LAYER_CREATOR(Convolution_final)
} // namespace ncnn
.........This file is included in the head of layer.cpp to provide the declaration of final class of layer and its associated creator for every layer used in the NCNN library.
.........
#ifdef __clang__
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Woverloaded-virtual"
#endif
#include "layer_declaration.h"
#ifdef __clang__
#pragma clang diagnostic pop
#endif
.........This file is generated using the similar way like the layer_declaration.h using Cmake configuration. It contains the body part of the registry table. If the NCNN_STRING is enabled, each entry pair contains the final class of the layer ##name_final and its associated creater ##name_final_layer_creater. Otherwise, only the associated creator is included.
......
#if NCNN_STRING
{"Convolution",Convolution_final_layer_creator},
#else
{Convolution_final_layer_creator},
#endif
......The file is included in the layer.cpp to complete the registry table of factory.
static const layer_registry_entry layer_registry[] =
{
#include "layer_registry.h"
};This file is generated using the similar way like the layer_declaration.h using Cmake configuration. It contains the class name and its enum value.
......
Convolution = 6,
......The file is included in the layer_type.h to complete the enum in LayerType. It is not related to the factory pattern directory (need to dig more detail).
namespace LayerType {
enum
{
#include "layer_type_enum.h"
CustomBit = (1<<8),
};
} // namespace LayerType
} // namespace ncnn