[oe-commits] org.oe.dev mono 1.2.5.1: added mono.bbclass, many changes required for packaging
henryk commit
openembedded-commits at lists.openembedded.org
Tue Oct 2 19:02:29 UTC 2007
mono 1.2.5.1: added mono.bbclass, many changes required for packaging
New file: packages/mono/mono-mcs-intermediate_1.2.5.1.bb
Compiles mono in native mode with standard prefix, then tars up the
resulting tree and puts the tarfile into staging
New file: packages/mono/mono_files.py
Automatically generated using collect-path.py (attached to this mail)
and contains a list that maps file patterns to package names (and
contained assemblies, see below).
New file: classes/mono.bbclass
Has a helper function for the list that maps file patterns to package
names and assemblies (see below). Also has a function mono_do_clilibs
and inserts that function into PACKAGEFUNCS. This function calls
mono_find_provides_and_requires which finds out (through calls to
monodis --assembly and monodis --assemblyref) which assemblies are
provided and required by a particular package. mono_do_clilibs then
puts the information about provided assemblies into
${STAGING_DIR}/clilibs/${packagename}.list and information about the
required packages into ${PKGDEST}/{packagename}.clilibdeps where it
will later be picked up by the modified read_shlibdeps.
Originally I had dependency resolution through the partial list in
mono_files.py but obviously this doens't scale, so I implemented the
new method with mono_do_clilibs. The benefit is now that I don't really
need the extra information in mono_files.py anymore and can in
principle get rid of mono_get_file_table and related code. Instead it
should be possible to modify collect-paths.py to output bitbake .inc
code (e.g. PACKAGES = "..." and a whole lot of FILES_... = "...")
instead of python code. There's still the minor problem of how to
handle the .mdb files, that's why I didn't implement it yet but instead
opted for an approach that I knew would work. (Debian just puts
the .mdb files into the individual packages, while I would argue that
they do belong into corresponding -dbg packages.)
Modified file: classes/package.bbclass
In read_shlibdeps I folded the two identical code blocks dealing with
*.shlibdeps and *.pcdeps into one and added *.clilibdeps (generated by
mono_do_clilibs above).
Modified file: packages/mono/mono_1.2.5.1.bb
Add the mono-mcs-intermediate workaround. Add a whole lot of python
code in populate_packages_prepend in order to split up the packages
based on information from mono_files.py (via mono.bbclass'
mono_get_file_table). As I said above a lot of this code can hopefully
be replaced in the future.
Author: henryk at openmoko.org
Branch: org.openembedded.dev
Revision: 7604e2e8a0c76d28ec378a7800c3e1a42b335da5
ViewMTN: http://monotone.openembedded.org/revision/info/7604e2e8a0c76d28ec378a7800c3e1a42b335da5
Files:
1
classes/mono.bbclass
packages/mono/collect-paths.py
packages/mono/mono-mcs-intermediate_1.2.5.1.bb
packages/mono/mono_files.py
classes/package.bbclass
packages/mono/README
packages/mono/mono_1.2.5.1.bb
Diffs:
#
# mt diff -rdf94b58a562e4f0ff6ffa655ec7dba895833e9bd -r7604e2e8a0c76d28ec378a7800c3e1a42b335da5
#
#
#
# add_file "classes/mono.bbclass"
# content [8c6f975d1387bb98881133402216adc4c3dcaac8]
#
# add_file "packages/mono/collect-paths.py"
# content [99c3e3ae6759faa7534a167d2460b09effd4bcbc]
#
# add_file "packages/mono/mono-mcs-intermediate_1.2.5.1.bb"
# content [cfb4d3805aa5678e130a2fe308b7bc1c5c8a9ded]
#
# add_file "packages/mono/mono_files.py"
# content [94dcfc9399594f824dcd8f036ce3d182d7193b16]
#
# patch "classes/package.bbclass"
# from [f055d1bce4ae74cd8130f0a0cd9c819aabc46c99]
# to [c40a90879037567f4894a6044537289dc71ffe3c]
#
# patch "packages/mono/README"
# from [8785ebebeb5995aafc71ea1e5c86e979a698470a]
# to [faa418e9818e111d50c8516b915b7f627fda79c9]
#
# patch "packages/mono/mono_1.2.5.1.bb"
# from [9499f337192480513d8f5e4e93ed53b3fe0f63ff]
# to [fcfef733807a03e005b267996fee10306b1d8df0]
#
============================================================
--- classes/mono.bbclass 8c6f975d1387bb98881133402216adc4c3dcaac8
+++ classes/mono.bbclass 8c6f975d1387bb98881133402216adc4c3dcaac8
@@ -0,0 +1,214 @@
+def mono_get_file_table(packageversion, d):
+ # The packageversion is currently ignored, but might be used in the future
+ # if more than one mono version is available and different versions
+ # need to use different tables
+
+ import bb, sys, os, glob, commands
+ curdir = os.path.dirname( bb.data.getVar('FILE', d, 1) )
+ if curdir not in sys.path: sys.path.append( curdir )
+ from mono_files import debian_mono_file_table
+
+ # mono-jay is not being built (for all platforms at least)
+ IGNORE = ("mono-jay", )
+ file_table = [
+ # Standard package
+ {"name": "mono-doc"},
+
+ # Virtual packages
+ {"name": "mono"},
+ {"name": "mono-runtime"},
+
+ # Not provided by Debian:
+ {"name": "libnunit2.2-cil",
+ "patterns": [
+ "/usr/lib/mono/gac/nunit.*/2.2.*",
+ "/usr/lib/mono/1.0/nunit.*.dll",
+ "/usr/lib/pkgconfig/mono-nunit.pc",
+ ],
+ "assemblies": [
+ ("nunit.core", "2.2.0.0"),
+ ("nunit.framework", "2.2.0.0"),
+ ("nunit.util", "2.2.0.0"),
+ ("nunit.mocks", "2.2.8.0"),
+ ],
+ },
+ {"name": "libmono-cecil0.5-cil",
+ "patterns": [
+ "/usr/lib/mono/gac/Mono.Cecil/0.5.*",
+ ],
+ "assemblies": [
+ ("Mono.Cecil", "0.5.*"),
+ ],
+ },
+ {"name": "libmono-db2-1.0-cil",
+ "patterns": [
+ "/usr/lib/mono/gac/IBM.Data.DB2/1.0*",
+ "/usr/lib/mono/1.0/IBM.Data.DB2.dll",
+ ],
+ "assemblies": [
+ ("IBM.Data.DB2", "1.0*"),
+ ],
+ },
+ ] + debian_mono_file_table
+
+ file_table = [e for e in file_table
+ if not (e.has_key("name") and e["name"] in IGNORE)]
+
+ return file_table
+
+def mono_find_provides_and_requires(files, d):
+ provides = []
+ requires = []
+
+ import bb, os, commands
+
+ pathprefix = "export PATH=%s; export LANG=; export LC_ALL=; " % bb.data.getVar('PATH', d, 1)
+ for filename in files:
+ if not filename.endswith(".dll") and not filename.endswith(".exe"):
+ continue
+ if not os.path.isfile(filename) or os.path.islink(filename):
+ continue
+
+ ## Provides
+ name, version = None, None
+
+ ret, result = commands.getstatusoutput("%smonodis --assembly '%s'" % (pathprefix, filename))
+ if ret:
+ bb.error("raw_provides_and_requires: monodis --assembly '%s' failed, dependency information will be inaccurate" % filename)
+ continue
+ for line in result.splitlines():
+ if not ":" in line: continue
+ key, value = line.split(":", 1)
+ if key.strip() == "Name":
+ name = value.strip()
+ elif key.strip() == "Version":
+ version = value.strip()
+ if name is not None and version is not None:
+ if (name, version) not in provides:
+ provides.append( (name, version) )
+
+ ## Requires
+ name, version = None, None
+ ret, result = commands.getstatusoutput("%smonodis --assemblyref '%s'" % (pathprefix, filename))
+ if ret:
+ bb.error("raw_provides_and_requires: monodis --assemblyref '%s' failed, dependency information will be inaccurate" % filename)
+ continue
+ for line in result.splitlines():
+ if not "=" in line: continue
+ key, value = line.split("=", 1)
+ if ":" in key and key.split(":",1)[1].strip() == "Version":
+ version = value.strip()
+ elif key.strip() == "Name":
+ name = value.strip()
+ if name is not None and version is not None:
+ if (name, version) not in requires:
+ requires.append( (name, version) )
+ name, version = None, None
+
+ # Remove everything from requires that's already in provides as it's not actually required
+ # to be provided externally
+ requires = [e for e in requires if not e in provides]
+ return provides, requires
+
+python mono_do_clilibs() {
+ import bb, os, re, os.path
+
+ exclude_clilibs = bb.data.getVar('EXCLUDE_FROM_CLILIBS', d, 0)
+ if exclude_clilibs:
+ bb.note("not generating clilibs")
+ return
+
+ lib_re = re.compile("^lib.*\.so")
+ libdir_re = re.compile(".*/lib$")
+
+ packages = bb.data.getVar('PACKAGES', d, 1)
+
+ workdir = bb.data.getVar('WORKDIR', d, 1)
+ if not workdir:
+ bb.error("WORKDIR not defined")
+ return
+
+ staging = bb.data.getVar('STAGING_DIR', d, 1)
+ if not staging:
+ bb.error("STAGING_DIR not defined")
+ return
+
+ pkgdest = bb.data.getVar('PKGDEST', d, 1)
+
+ clilibs_dir = os.path.join(staging, "clilibs")
+ bb.mkdirhier(clilibs_dir)
+
+ provides, requires = {}, {}
+ private_libs = bb.data.getVar('PRIVATE_CLILIBS', d, 1)
+ for pkg in packages.split():
+ bb.debug(2, "calculating clilib provides for %s" % pkg)
+
+ files_to_check = []
+ top = os.path.join(pkgdest, pkg)
+ for root, dirs, files in os.walk(top):
+ for file in files:
+ path = os.path.join(root, file)
+ if file.endswith(".exe") or file.endswith(".dll"):
+ files_to_check.append( path )
+ provides[pkg], requires[pkg] = mono_find_provides_and_requires(files_to_check, d)
+ clilibs_file = os.path.join(clilibs_dir, pkg + ".list")
+ if os.path.exists(clilibs_file):
+ os.remove(clilibs_file)
+ if len(provides[pkg]) > 0:
+ fd = open(clilibs_file, 'w')
+ for s in provides[pkg]:
+ fd.write(" ".join(s) + '\n')
+ fd.close()
+
+ clilib_provider = {}
+ list_re = re.compile('^(.*)\.list$')
+ for file in os.listdir(clilibs_dir):
+ m = list_re.match(file)
+ if m:
+ dep_pkg = m.group(1)
+ fd = open(os.path.join(clilibs_dir, file))
+ lines = fd.readlines()
+ fd.close()
+ for l in lines:
+ clilib_provider[tuple(l.rstrip().split())] = dep_pkg
+
+ for pkg in packages.split():
+ bb.debug(2, "calculating clilib requirements for %s" % pkg)
+
+ deps = []
+ for n in requires[pkg]:
+ if n in clilib_provider.keys():
+ dep_pkg = clilib_provider[n]
+
+ if dep_pkg == pkg:
+ continue
+
+ if not dep_pkg in deps:
+ deps.append(dep_pkg)
+ else:
+ bb.note("Couldn't find CLI library provider for %s" % n)
+
+ deps_file = os.path.join(pkgdest, pkg + ".clilibdeps")
+ if os.path.exists(deps_file):
+ os.remove(deps_file)
+ if len(deps) > 0:
+ fd = open(deps_file, 'w')
+ for dep in deps:
+ fd.write(dep + '\n')
+ fd.close()
+}
+
+python() {
+ # Insert mono_do_clilibs into PACKAGEFUNCS
+ # Needs to be called after populate_packages, but before read_shlibdeps
+ PACKAGEFUNCS = bb.data.getVar("PACKAGEFUNCS", d, 1)
+ if PACKAGEFUNCS:
+ PACKAGEFUNCS = PACKAGEFUNCS.split()
+ if "read_shlibdeps" in PACKAGEFUNCS:
+ i = PACKAGEFUNCS.index("read_shlibdeps")
+ PACKAGEFUNCS.insert(i, "mono_do_clilibs")
+ elif "populate_packages" in PACKAGEFUNCS:
+ i = PACKAGEFUNCS.index("populate_packages")
+ PACKAGEFUNCS.insert(i+1, "mono_do_clilibs")
+ bb.data.setVar("PACKAGEFUNCS", " ".join(PACKAGEFUNCS), d)
+}
============================================================
--- packages/mono/collect-paths.py 99c3e3ae6759faa7534a167d2460b09effd4bcbc
+++ packages/mono/collect-paths.py 99c3e3ae6759faa7534a167d2460b09effd4bcbc
@@ -0,0 +1,135 @@
+#!/usr/bin/env python
+
+## This utility takes the debian directory from an unpacked debian mono source tree
+## (e.g. apt-get source mono), parses the *.install files and generates python source
+## for a list of dictionaries that describe the individual packages and their contents
+## The output will look like
+##debian_mono_file_table = [
+## { 'name': 'libmono-peapi1.0-cil',
+## 'patterns': [
+## '/usr/lib/mono/gac/PEAPI/1.0.*/',
+## '/usr/lib/mono/1.0/PEAPI.dll'
+## ],
+## 'assemblies': [
+## ('PEAPI', '1.0.*')
+## ]
+## },
+## { 'name': 'mono-mjs',
+## 'patterns': [
+## '/usr/bin/mjs',
+## '/usr/lib/mono/1.0/mjs.exe*'
+## ]
+## },
+##....
+
+
+import os, sys, re
+
+def collect_paths(dir):
+ paths = {}
+
+ os.chdir(dir)
+ for filename in os.listdir("."):
+ if filename.endswith(".install"):
+ fp = file(filename, "r")
+ lines = fp.readlines()
+ fp.close()
+
+ contents = []
+ for line in lines:
+ lineparts = line.strip().split()
+ if lineparts[0].startswith("debian/tmp"):
+ pattern = lineparts[0][ len("debian/tmp"): ]
+ if len(lineparts) == 2:
+ if not pattern.startswith(lineparts[1]):
+ print >>sys.stderr, "Warning: Apparently I don't fully understand the format in file %s" % filename
+ elif len(lineparts) > 2:
+ print >>sys.stderr, "Warning: Apparently I don't fully understand the format in file %s" % filename
+
+ contents.append( pattern )
+ else:
+ print >>sys.stderr, "Note: Ignoring %s in %s" % (lineparts, filename)
+
+ paths[ filename[ :-len(".install") ] ] = contents
+
+ return paths
+
+def collect_packages(paths):
+ gac_re = re.compile(r'/usr/lib/mono/gac/(?P<assembly>[^/]+)/(?P<version>[^/]+)/?')
+
+ # These packages should be populated first (e.g. because their files will otherwise end up
+ # in other packages)
+ PACKAGES_FIRST = ("mono-jit", "mono-gac", "mono-mjs", "mono-gmcs", "mono-utils", "mono-doc")
+ # These should be populated last (because their spec is very broad)
+ PACKAGES_LAST = ("mono-mcs", "libmono-system1.0-cil", "libmono-system2.0-cil", "libmono1.0-cil", "libmono2.0-cil")
+ first = []
+ last = []
+ packages = paths.keys()
+ for packagename in PACKAGES_FIRST + PACKAGES_LAST:
+ if packagename in packages:
+ packages.remove(packagename)
+ if packagename in PACKAGES_FIRST:
+ first.append(packagename)
+ else:
+ last.append(packagename)
+ packagenames = first + packages + last
+
+ packages = []
+ for name in packagenames:
+ patterns = paths[ name ]
+ package = { "name": name,
+ "patterns": patterns}
+
+ provided_assemblies = []
+ for pattern in patterns:
+ match = gac_re.match(pattern)
+ if match:
+ provided_assemblies.append( (match.group("assembly"), match.group("version")) )
+ if pattern == "/usr/lib/mono/1.0/mscorlib.dll*":
+ provided_assemblies.append( ("mscorlib", "1.0.*" ) )
+ elif pattern == "/usr/lib/mono/2.0/mscorlib.dll*":
+ provided_assemblies.append( ("mscorlib", "2.0.*" ) )
+
+ if len(provided_assemblies) > 0:
+ package["assemblies"] = provided_assemblies
+
+ packages.append(package)
+
+ return packages
+
+if __name__ == "__main__":
+ packages = collect_packages( collect_paths(".") )
+
+ if False: # Human-friendly
+ for package in packages:
+ print "Package: %s" % package["name"]
+ if package.has_key("provided_assemblies"):
+ print "Provides: \t%s" % ( "\n\t\t".join( [" ".join(e) for e in package["assemblies"] ] ) )
+ print "Patterns: \t\t%s" % ( "\n\t\t\t".join(package["patterns"]) )
+ print
+ else:
+ print "# This is a generated file, please do not edit directly"
+ print "# Use collect-paths.py instead. -- Henryk <henryk at openmoko.org>"
+ print "debian_mono_file_table = ["
+ print ",\n".join(
+ [
+ "\t{\t%s\n\t}" % ",\n\t\t".join(
+ [
+ "%r: %r" % (key, value)
+ for key, value in package.items()
+ if not isinstance(value, (list,tuple))
+ ] + [
+ "%r: [\n\t\t\t\t%s\n\t\t\t]" % (key, ",\n\t\t\t\t".join( [
+ "%r"%(e,) for e in value
+ ])
+ )
+ for key, value in package.items()
+ if isinstance(value, (list,tuple))
+ ]
+
+ )
+ for package in packages
+ ]
+ )
+ print "]"
+
============================================================
--- packages/mono/mono-mcs-intermediate_1.2.5.1.bb cfb4d3805aa5678e130a2fe308b7bc1c5c8a9ded
+++ packages/mono/mono-mcs-intermediate_1.2.5.1.bb cfb4d3805aa5678e130a2fe308b7bc1c5c8a9ded
@@ -0,0 +1,59 @@
+# This is a straw-man recipe for step 1 in the two-step build of
+# mono. Because it's impossible to build the mcs directory
+# in cross-compile mode, this recipe will do a native build,
+# then tar the resulting install tree for usage by the mono
+# package in step 2.
+# See http://www.mono-project.com/Mono:ARM
+
+require mono_1.2.5.inc
+PR = "r0"
+DEPENDS = "mono-native glib-2.0-native"
+
+SRC_URI += "file://mono-fix-libdir-path.patch;patch=1"
+
+# Inherit native to set up compiler and paths ...
+inherit native
+# ... but override the target prefix
+prefix = "/usr"
+exec_prefix = "/usr"
+sysconfdir = "/etc"
+# TODO: Where does the mono package get
+# these paths from? Use the same source.
+
+do_fix_libtool_name() {
+ # inherit native will make that all native tools that are being
+ # built are prefixed with something like "i686-linux-",
+ # including libtool. Fix up some hardcoded libtool names:
+ for i in "${S}"/runtime/*-wrapper.in; do
+ sed -e "s/libtool/${BUILD_SYS}-libtool/" -i "${i}"
+ done
+}
+addtask fix_libtool_name after do_patch before do_configure
+
+do_stage() {
+ true
+}
+
+do_install() {
+ oe_runmake 'DESTDIR=${D}' install
+}
+
+do_package() {
+ true
+}
+
+do_populate_staging() {
+ cd ${D}
+ rm -f ${WORKDIR}/mono-mcs-${PV}.tar.gz
+ tar -cvzf ${WORKDIR}/mono-mcs-${PV}.tar.gz .
+ install -d ${STAGING_DIR}/share/mono-mcs
+ cp ${WORKDIR}/mono-mcs-${PV}.tar.gz ${STAGING_DIR}/share/mono-mcs/
+}
+
+do_package_write_ipk() {
+ true
+}
+
+do_package_write() {
+ true
+}
============================================================
--- packages/mono/mono_files.py 94dcfc9399594f824dcd8f036ce3d182d7193b16
+++ packages/mono/mono_files.py 94dcfc9399594f824dcd8f036ce3d182d7193b16
@@ -0,0 +1,605 @@
+# This is a generated file, please do not edit directly
+# Use collect-paths.py instead. -- Henryk <henryk at openmoko.org>
+debian_mono_file_table = [
+ { 'name': 'mono-jit',
+ 'patterns': [
+ '/usr/bin/mono'
+ ]
+ },
+ { 'name': 'mono-gac',
+ 'patterns': [
+ '/usr/bin/gacutil',
+ '/usr/lib/mono/1.0/gacutil.exe'
+ ]
+ },
+ { 'name': 'mono-mjs',
+ 'patterns': [
+ '/usr/bin/mjs',
+ '/usr/lib/mono/1.0/mjs.exe*'
+ ]
+ },
+ { 'name': 'mono-gmcs',
+ 'patterns': [
+ '/usr/bin/gmcs',
+ '/usr/bin/wsdl2',
+ '/usr/bin/monop2',
+ '/usr/bin/ilasm2',
+ '/usr/bin/resgen2',
+ '/usr/bin/mono-api-info2',
+ '/usr/bin/mono-service2',
+ '/usr/bin/mkbundle2',
+ '/usr/bin/xbuild',
+ '/usr/bin/sgen',
+ '/usr/bin/al2',
+ '/usr/bin/httpcfg',
+ '/usr/lib/mono/2.0/*.exe*',
+ '/usr/lib/mono/2.0/xbuild.rsp',
+ '/usr/lib/mono/2.0/MSBuild/',
+ '/usr/lib/mono/2.0/Microsoft.Build.xsd',
+ '/usr/lib/mono/2.0/Microsoft.CSharp.targets',
+ '/usr/lib/mono/2.0/Microsoft.Common.tasks',
+ '/usr/lib/mo%s
>>> DIFF TRUNCATED @ 16K
More information about the Openembedded-commits
mailing list