Passive Component Registration
Motivation
UNO components in the various language bindings (dynamic libraries for binary/C++ UNO, jars for Java UNO, etc.) have mechanisms to actively report information about the services and singletons they offer (exported symbol component_writeInfo
for dynamic libraries, public static __writeRegistryServiceInfo
method for jars, etc.). This active component registration has various drawbacks:
- Calls to actively register the components (
regcomp
) need to be made in carefully crafted environments, ensuring that the relevant UNO components can actually be loaded and their code executed. Especially for Java and Python components, this is nontrivial.- At OOo build time, when a
services.rdb
is assembled ininstsetoo_native
. - At OOo installation time, when bundled extensions that contain UNO components are registered.
- At OOo run time, when extensions that contain UNO components are registered or revoked.
- At OOo build time, when a
- Loading the UNO components to actively execute their code can be time consuming (especially if the components' functionality is not otherwise needed, like when assembling a
services.rdb
at build time).
Therefore, it is sought to replace active component registration with a passive approach, where a UNO component is accompanied by a representation of the relevant information that can be obtained without running the component's code.
An XML Format
The canonic choice for representing such information these days is XML. The necessary information is encoded as follows:
An XML document encoding information about multiple UNO components (i.e., corresponding to a services.rdb
) has a root element with local name components
and namespace URI http://openoffice.org/2010/uno-components
.
The components
root element contains zero or more elements with local name component
and namespace URI http://openoffice.org/2010/uno-components
, each with an attribute with unprefixed name uri
, the value of which is a URI (in OOo-internal form) denoting the relevant UNO component (and which may be a vnd.sun.star.expand
URL), and an attribute with unprefixed name loader
, the value of which is the name of the relevant UNO service with which to load the component (e.g., com.sun.star.loader.SharedLibrary
or com.sun.star.loader.Java2
). Each component
element in turn contains zero or more service
elements, followed by zero or more singleton
elements.
Each element with local name service
or singleton
and namespace URI http://openoffice.org/2010/uno-components
has an attribute with unprefixed name name
, the value of which is the name of the respective service or singleton.
(The information provided by active component registration contains redundancies that are dropped from this XML format. For one, UNO components are listed in an IMPLEMENTATIONS
section and each service and singleton is again listed in a SERVICES
resp. SINGLETONS
section. For another, each singleton specifies the service implementing it.)
There was already a somewhat similar textual format for regcomplazy
in use, which however was based on ini-file syntax (but with irregularities, containing lines that do not match the key=value
format) instead of XML. This has been superseded by the approach described here, and regcomplazy
has been removed.
As we will see later, within the OOo build system there are also XML files describing single UNO components, and which have a component
element instead of a components
element as their root element, but otherwise use the same structure as the above XML format.
XML at Run Time
There was a choice whether to either take the XML files and inject them into registry-format services.rdb
files (i.e., to still call regcomp
resp. unopkg
at build resp. installation time), or to use the XML files directly at run time. I decided for the latter.
The configmgr
re-write showed that it is acceptable performance-wise to read a handful of XML files during OOo start up, using an XML reader specifically written for that task. So it should be acceptable to replace the existing registry-format services.rdb
files (for a start, the URE layer services.rdb
and the basis layer services.rdb
and legacy_binfilters.rdb
) with XML files.
The advantages are easier and potentially faster creation (no need for regcomp
like code any more), and easier manipulation (due to the textual format). Also, the registry format has a known design defect, in that it cannot handle arbitrary combinations of key names.
Work in Progress
Work is done on CWS sb129 , currently as a stack of MQ patches:
simpleregistry-cleanup
cleans up the stoc/source/simpleregistry
code so that subsequent patches can better modify it.
xmlreader
extracts the XmlReader
code from configmgr
to a new xmlreader
URE module, so that it can be reused from stoc
. An open problem is how to version its C++ ABI (currently done via symbol visibility, which does not allow for versioning).
textualservices
extends the implementation of com.sun.star.registry.SimpleRegistry
in stoc/source/simpleregistry
, so that it can internally also read XML files in addition to registry files, and present them to its clients as if they had the structure of registry-format services.rdb
files. This appeared to be the simplest approach for now, given how deep the knowledge about registry-format files is rooted in the code base (see, for example, the functionality in cppuhelper/bootstrap.hxx
that bootstraps a UNO environment based on rdb
files); it is still subject to change, however.
passive
replaces the active registration of all of OOo's UNO components with passive registration. The resulting XML files are still called services.rdb
etc. (instead of, say, services.xml
), mainly because [ure/source/README
] declares the URE services.rdb
as part of the published URE interface (acknowledging only its existence, not its internal format, and especially warning against registering anything into it).
.component
Files
For each UNO component implementation in the OOo code base that does not implement a UNO component that is included in backwards-compatible extensions (see below), any existing active registration code is removed, and instead a X.component
file is added. That file describes component X
in the above XML format, with a component
root element that lacks the uri
attribute (because it is platform dependent, e.g., libvclli.so
vs. vclmi.dll
). Then, a makefile.mk
rule uses new solenv/bin/createcomponent.xslt
to turn this into a proper .component
file including uri
. New postprocess/packcomponents/makefile.mk
uses new solenv/bin/packcomponents.xslt
to bundle those .component
files into the basis-layer services.rdb
(similarly, URE-layer services.rdb
is assembled in ure/source/makefile.mk
and basis-layer legacy_binfilters.rdb
is assembled in binfilter/util/makefile.mk
).
The corresponding information about UNO component registration was removed from the scp2
data. One problem is that what components were actually registered into a registry-format services.rdb
in instsetoo_native
depended on what scp2
File
definitions are visible when building a product. This varies among products (especially between OpenOffice.org and Oracle Open Office), so that different products used to silently produce different basis-layer services.rdb
files (even though this must not be). With the new, more static approach of assembling services.rdb
files, this needs to be addressed properly (see TODO
s in
postprocess/packcomponents/makefile.mk
).
The below throwaway Haskell script was used to translate the
regview
output of an existing registry-format services.rdb
into XML format, copying the resulting component
elements into the individual .component
files:
-- Makefile:
-- .PHONY: components
-- components:
-- ghc --make -o $@ components.hs
--
-- Extract components XML data from regview output:
--
-- $ regview ... | components > ...
{-# LANGUAGE ScopedTypeVariables #-}
module Main (main) where
import Data.Map (Map, alter, empty, foldWithKey, insert, singleton, toList)
import Data.List (sort)
import Text.Regex.Posix ((=~))
type Impls = Map String Impl -- keyed on implementation name
data Impl =
Impl { activator :: String, location :: String, services :: [String],
singletons :: [String] }
deriving Show
type Comps = Map String Comp -- keyed on URI
data Comp = Comp { loader :: String, impls :: Impls }
deriving Show
data State = StateStart | StateImplementations | StateImplementation String
| StateActivator String | StateLocation String | StateServices String
| StateSingletons String
(=~~~) :: String -> String -> Maybe [String]
a =~~~ b = if c /= "" then Just ds else Nothing
where (_::String, c::String, _::String, ds) = a =~ b
getImpl :: Maybe Impl -> Impl
getImpl (Just impl) = impl
getImpl Nothing = Impl undefined undefined [] []
setActivator :: Impls -> String -> String -> Impls
setActivator c id n = alter alt id c
where alt x = let i = getImpl x in Just $ i { activator = n }
setLocation :: Impls -> String -> String -> Impls
setLocation c id n = alter alt id c
where alt x = let i = getImpl x in Just $ i { location = n }
addService :: Impls -> String -> String -> Impls
addService c id n = alter alt id c
where alt x = let i = getImpl x in Just $ i { services = n : services i }
addSingleton :: Impls -> String -> String -> Impls
addSingleton c id n = alter alt id c
where alt x = let i = getImpl x
in Just $ i { singletons = n : singletons i }
parseLine :: [State] -> Impls -> IO ([State], Impls)
parseLine s c = do l <- getLine
return $ parseLine' s l
where
parseLine' ss@(_ : sr) l =
case parseLine'' ss l of
Just a -> a
_ -> parseLine' sr l
parseLine' [] l = (s, c)
parseLine'' ss@(StateStart : _) l =
if l == " / IMPLEMENTATIONS" then Just (StateImplementations : ss, c)
else Nothing
parseLine'' ss@(StateImplementations : _) l =
case l =~~~ "^ / (.+)$" of
Just [id] -> Just (StateImplementation id : ss, c)
_ -> case l =~~~ "^ / " of
Just _ -> Just ([], c)
_ -> Nothing
parseLine'' ss@(StateImplementation id : _) l =
if l == " / ACTIVATOR" then Just (StateActivator id : ss, c)
else if l == " / LOCATION" then Just (StateLocation id : ss, c)
else if l == " / SERVICES"
then Just (StateServices id : ss, c)
else if l == " / SINGLETONS"
then Just (StateSingletons id : ss, c)
else if l == " / Loader"
then Just (ss, c)
else Nothing
parseLine'' ss@(StateActivator id : _) l =
case l =~~~ "^ Data = \"(.+)\"$" of
Just [n] -> Just (ss, setActivator c id n)
_ -> Nothing
parseLine'' ss@(StateLocation id : _) l =
case l =~~~ "^ Data = \"(.+)\"$" of
Just [n] -> Just (ss, setLocation c id n)
_ -> Nothing
parseLine'' ss@(StateServices id : _) l =
case l =~~~ "^ / (.+)$" of
Just [n] -> Just (ss, addService c id n)
_ -> Nothing
parseLine'' ss@(StateSingletons id : _) l =
case l =~~~ "^ / (.+)$" of
Just [n] -> Just (ss, addSingleton c id n)
_ -> Nothing
parse :: IO Impls
parse = parse' [StateStart] empty
where parse' s c = do (s', c') <- parseLine s c
if null s' then return c' else parse' s' c'
toComps :: Impls -> Comps
toComps = foldWithKey toComp empty
where toComp id impl cs = alter alt (location impl) cs
where alt Nothing =
Just $ Comp { loader = activator impl,
impls = singleton id impl }
alt (Just c) | loader c == activator impl =
Just $ c { impls = insert id impl $ impls c }
xmlComponent :: (String, Comp) -> IO ()
xmlComponent (uri, comp) = do
putStrLn $
" <component loader=\"" ++ loader comp ++ "\" uri=\"" ++ uri ++ "\">"
mapM_ xmlImplementation $ toList $ impls comp
putStrLn " </component>"
xmlImplementation :: (String, Impl) -> IO ()
xmlImplementation (id, impl) = do
putStrLn $ " <implementation name=\"" ++ id ++ "\">"
mapM_ xmlService $ sort $ services impl
mapM_ xmlSingleton $ sort $ singletons impl
putStrLn " </implementation>"
xmlService :: String -> IO ()
xmlService n = putStrLn $ " <service name=\"" ++ n ++ "\"/>"
xmlSingleton :: String -> IO ()
xmlSingleton n = putStrLn $ " <singleton name=\"" ++ n ++ "\"/>"
main :: IO ()
main = do impls <- parse
let comps = toComps impls
putStrLn "<?xml version=\"1.0\"?>"
putStrLn "<components xmlns=\"http://openoffice.org/2010/uno-components\">"
mapM_ xmlComponent $ toList comps
putStrLn "</components>"
That nothing broke during translation can be verified (within the Hamburg environment) with the below throwaway Bash script:
#!/bin/bash
set -e -o pipefail
mkdir out
for i in {unxlngi6,unxsoli4,unxsols4}{,.pro} unxlngx6.pro; do
LD_LIBRARY_PATH=/so/ws/DEV300/unxlngi6.pro/lib.m87 /so/ws/DEV300/unxlngi6.pro/bin.m87/regview /so/ws/DEV300/$i/installation.m87/opt/openoffice.org/ure/share/misc/services.rdb > out/regview-DEV300_m87-$i-ure-services.rdb
LD_LIBRARY_PATH=/so/ws/DEV300/unxlngi6.pro/lib.m87 /so/ws/DEV300/unxlngi6.pro/bin.m87/regview /so/ws/DEV300/$i/installation.m87/opt/openoffice.org/basis3.4/program/services.rdb > out/regview-DEV300_m87-$i-basis-services.rdb
LD_LIBRARY_PATH=/so/ws/DEV300/unxlngi6.pro/lib.m87 /so/ws/DEV300/unxlngi6.pro/bin.m87/regview /so/ws/DEV300/$i/installation.m87/opt/openoffice.org/basis3.4/program/legacy_binfilters.rdb > out/regview-DEV300_m87-$i-basis-legacy_binfilters.rdb
done
for i in unxmacxi{,.pro}; do
LD_LIBRARY_PATH=/so/ws/DEV300/unxlngi6.pro/lib.m87 /so/ws/DEV300/unxlngi6.pro/bin.m87/regview /so/ws/DEV300/$i/installation.m87/opt/OpenOffice.org.app/Contents/basis-link/ure-link/share/misc/services.rdb > out/regview-DEV300_m87-$i-ure-services.rdb
LD_LIBRARY_PATH=/so/ws/DEV300/unxlngi6.pro/lib.m87 /so/ws/DEV300/unxlngi6.pro/bin.m87/regview /so/ws/DEV300/$i/installation.m87/opt/OpenOffice.org.app/Contents/basis-link/program/services.rdb > out/regview-DEV300_m87-$i-basis-services.rdb
LD_LIBRARY_PATH=/so/ws/DEV300/unxlngi6.pro/lib.m87 /so/ws/DEV300/unxlngi6.pro/bin.m87/regview /so/ws/DEV300/$i/installation.m87/opt/OpenOffice.org.app/Contents/basis-link/program/legacy_binfilters.rdb > out/regview-DEV300_m87-$i-basis-legacy_binfilters.rdb
done
for i in wntmsci12{,.pro}; do
mkdir tmp
(cd tmp && unzip /so/install/$i/OpenOffice/archive/DEV300_m87_native_packed-1_en-US.9522/OOo_3.4.0_Win_x86_install-arc_en-US.zip)
LD_LIBRARY_PATH=/so/ws/DEV300/unxlngi6.pro/lib.m87 /so/ws/DEV300/unxlngi6.pro/bin.m87/regview $PWD/tmp/OOo_3.4.0_Win_x86_install-arc_en-US/OpenOffice.org\ 3/URE/misc/services.rdb > out/regview-DEV300_m87-$i-ure-services.rdb
LD_LIBRARY_PATH=/so/ws/DEV300/unxlngi6.pro/lib.m87 /so/ws/DEV300/unxlngi6.pro/bin.m87/regview $PWD/tmp/OOo_3.4.0_Win_x86_install-arc_en-US/OpenOffice.org\ 3/Basis/program/services.rdb > out/regview-DEV300_m87-$i-basis-services.rdb
LD_LIBRARY_PATH=/so/ws/DEV300/unxlngi6.pro/lib.m87 /so/ws/DEV300/unxlngi6.pro/bin.m87/regview $PWD/tmp/OOo_3.4.0_Win_x86_install-arc_en-US/OpenOffice.org\ 3/Basis/program/legacy_binfilters.rdb > out/regview-DEV300_m87-$i-basis-legacy_binfilters.rdb
rm -r tmp
done
for i in out/regview-*; do
../haskell/components < "$i" > out/processed"${i#out/regview}"
done
for i in {unxlngi6,unxmacxi,unxsoli4,unxsols4,wntmsci12}{,.pro} unxlngx6; do
printf '\n%s ure-services:\n' $i
diff <(cat /cws/so-cwsserv02/sb129/DEV300/$i/xml/ure/services.rdb | sed -e 's/<component /\n<component /g' -e 's:</components>:\n</components>:' | grep '^<component ' | sort) <(cat out/processed-DEV300_m87-$i-ure-services.rdb | sed -e 's/^ *//' | tr -d '\n' | sed -e 's/<component /\n<component /g' -e 's:</components>:\n</components>:' | grep '^<component ' | sort) || [ $? -eq 1 ]
printf '\n%s basis-services:\n' $i
diff <(cat /cws/so-cwsserv02/sb129/DEV300/$i/xml/services.rdb | sed -e 's/<component /\n<component /g' -e 's:</components>:\n</components>:' | grep '^<component ' | sort) <(cat out/processed-DEV300_m87-$i-basis-services.rdb | sed -e 's/^ *//' | tr -d '\n' | sed -e 's/<component /\n<component /g' -e 's:</components>:\n</components>:' | grep '^<component ' | sort) || [ $? -eq 1 ]
printf '\n%s legacy_binfilters-services:\n' $i
diff <(cat /cws/so-cwsserv02/sb129/DEV300/$i/xml/legacy_binfilters.rdb | sed -e 's/<component /\n<component /g' -e 's:</components>:\n</components>:' | grep '^<component ' | sort) <(cat out/processed-DEV300_m87-$i-basis-legacy_binfilters.rdb | sed -e 's/^ *//' | tr -d '\n' | sed -e 's/<component /\n<component /g' -e 's:</components>:\n</components>:' | grep '^<component ' | sort) || [ $? -eq 1 ]
done
Extensions
For backwards compatibility, UNO components included in extensions that shall still work with older OOo versions must continue to use active registration. The components that are probably affected (and which have not been translated to passive registration above) are exactly those in modules mysqlc
, reportbuilder
, sdext
, and swext
.
How passive registration of UNO components in extensions will work is still open.
Removals
solenv/src/component.map
and solenv/src/unloadablecomponent.map
no longer mention component_writeInfo
(a new solenv/src/reg-component.map
is a copy of the original component.map
, including component_writeInfo
).
comphelper/servicedecl.hxx
no longer implements component_writeInfo
functionality.
regcomplazy
is gone.
STARREGISTRY
, UNO_COMPONENT
, RegistryID
, NativeServicesURLPrefix
, JavaServicesURLPrefix
, and Regmergefile
are gone from scp2
(and thus also UNO_JAR_FILE
, UNO_JAR_FILE_PATCH
, PACKED_UNO_LIB_FILE_BODY
, PACKED_UNO_LIB_FILE_BODY_PATCH
, STD_UNO_LIB_FILE
, STD_UNO_LIB_FILE_PATCH
, SPECIAL_UNO_LIB_FILE
, SPECIAL_UNO_LIB_FILE_PATCH
, SPECIAL_UNO_COMPONENT_LIB_FILE
, SPECIAL_UNO_COMPONENT_LIB_FILE_PATCH
, and SPECIAL_UNO_NO_WARNING_IF_NOT_EXISTS_FILE
from scp2/inc/macros.inc
; replacements are SPECIAL_COMPONENT_LIB_FILE
and SPECIAL_COMPONENT_LIB_FILE_PATCH
).
SERVICESPROJEKT
is gone from instsetoo_native
.