[OE-core] Directory Ownership - RFC

Fri Jun 10 20:32:08 UTC 2011

A follow-up with this.

For an RPM based system this should be fairly easy to implement.  However, after
doing investigation on deb and opkg (related), I'm not sure this is going to work.

>From my investigation, deb formatted packages have no concept of directory
ownership.  Directories just "exist" in the system.  Whoever creates the
directories sets the permissions, owner and group.  In Debian they have a number
of helpers that ensure the permissions/owner/group of directories are
reasonable, and if there is a mismatch it indicates a bug in the two packages
that has to be corrected.

So at this point, I'm not exactly sure how to continue.  If we decide to take
the approach of the Debian systems, we need to likely add some type of QA/Sanity
checking for directories to avoid the "sudo" problem mentioned below, as well as
add some helper functionality to ensure the base directory set is reasonable at
all times.

We could also do a hybrid approach, use the mechanism mentioned below or
something similar when using RPM packaging, but fall back to just including
everything for Deb / opkg and resolving mismatches when found.

Right now I'm leaning toward the hybrid approach.

 * Add tooling that ensures base directories always match a distro policy
   - Likely a distro set of owned directories with their modes, owners and groups
   - This would likely need to also be used by the base-files package as well

 * Add RPM packaging exceptions that base directories are ignored when packaging
(except when specifically included)
   - We would only own directories not in the distro list mentioned above -- or
when the feature was disabled.
   - For RPM this helps the installer better order the installation based on
directory dependencies and also eliminates some of the metadata that has to be
carried around

Any comments, suggestions?

On 6/6/11 4:32 PM, Mark Hatle wrote:
> I've spoken to a few people on this..  Revision and comments below in-line...
> 
> On 6/6/11 11:31 AM, Mark Hatle wrote:
>> I'm starting to work on an enhancement that will allow (binary) packages
>> produced from the recipes to clearly own directories.  I'm trying to find a
>> simple enough solution that avoids having to change most recipes, while still
>> giving the developer and distribution designers flexibility to establish clear
>> lines of ownership and system/distribution policy.
>>
>> Problem
>> -------
>>
>> Currently directories are "owned" by whatever packages use them.  This works for
>> the most part, but can lead to problems when many different packages use the
>> same directory -- such as /bin, /sbin, /etc, /lib, /usr/lib, etc....
>>
>> When packaging (at least with RPM), the directory permissions, owners and groups
>> are automatically included into the package using the existing FILES_pkg globbing.
>>
>> The primary issue is that the first package that creates/uses the directory is
>> generally the one that sets the permissions, owner and group.  For many
>> directories 0755, root, root is perfectly fine, however other standard
>> directories may require specific modes, owners and groups...  (We recently had
>> an issue with sudo creating /var and /var/lib as 0700... which caused failures
>> in other packages.)
>>
>> This causes issues with consistency between recipes as well as potential
>> security implications on multi-user systems.
>>
>> Proposed solution
>> -----------------
>>
> 
> Remove the items below
> 
>> Add a new "DIRS" variable, similar to the existing "FILES", that will specify
>> the directories for a package to own.  By default DIRS = FILES, i.e. DIRS_${PN}
>> ?= FILES_${PN}
>>
>> The purpose of this is to automatically inherit the files list and walk the
>> directories in the same way that existing recipes do it.  Resulting in no change
>> to most recipes, but also giving a package a clear way to override the values
>> and specify select directories to own.
>>
>> A key difference in behavior between the DIRS and FILES methods would be any
>> directory specified with a "/" at the end of the name would avoid walking the
>> directory.  For example, DIRS = /usr/share/foo  would include "foo" and all of
>> the directories under it.  While DIRS = /usr/share/foo/ would simply include
>> "foo", and avoid walking the subdirectories.  Q: Is this confusing behavior?
>>
>> In addition to the above, a new way to exclude directories from being included
>> -- even if in the DIRS list is needed to keep things "simple".
> 
> Remove the items above
> 
> This is thought to be way too confusing, as DIRS is the same as FILES 99% of the
> time.  Instead, the portion below on the EXCLUDEDIRS would be modified to work
> like "FILES".  It would attempt to use globbing to walk directories and such.
> 
> So if you didn't want all of the directories in say ${mandir}, you would add:
> 
> ${mandir}/*
> 
> to the EXCLUDEDIRS.
> 
> This would have the effect of excluding all directories in and under the
> ${mandir} path.
> 
> The only difference in behavior between FILES and EXCLUDEDIRS is that
> directories specified w/o globbing at the end would be treated as just THAT
> directory.  If there is globbing as the last item (ending with a "*"), then
> whatever is found, and those directories contained within will be excluded from
> ownership as well.
> 
>> This requires adding a new variable that that will contain a list of directories
>> that should be ignored when processing the FILES_pkg.  This exclusion list only
>> covers the directories themselves, not the files that would normally be found
>> inside of them.
>>
>> The exclusion variable would be set, by default, to (based on base-files):
>>
>> /tmp ${localstatedir}/volatile/lock ${localstatedir}/volatile/tmp
>> /home ${prefix}/src ${localstatedir}/local
>> /bin /boot /dev ${sysconfdir} ${sysconfdir}/default
>> ${sysconfdir}/skel /lib /mnt /proc /home/root /sbin
>> ${prefix} ${bindir} ${docdir} /usr/games ${includedir}
>> ${libdir} ${sbindir} ${datadir}
>> ${datadir}/common-licenses ${datadir}/dict ${infodir}
>> ${mandir} ${datadir}/misc ${localstatedir}
>> ${localstatedir}/backups ${localstatedir}/lib
>> /sys ${localstatedir}/lib/misc ${localstatedir}/spool
>> ${localstatedir}/volatile ${localstatedir}/volatile/cache
>> ${localstatedir}/volatile/lock/subsys
>> ${localstatedir}/volatile/log
>> ${localstatedir}/volatile/run
>> /mnt /media /media/card /media/cf /media/net /media/ram
>> /media/union /media/realroot /media/hdd
>> /media/mmc1
>> /srv
>> ${prefix}/local ${prefix}/local/bin ${prefix}/local/games
>> ${prefix}/local/include ${prefix}/local/lib ${prefix}/local/sbin
>> ${prefix}/local/share ${prefix}/local/src
>> /var/mail
>>
>> Assuming the variable is called "EXCLUDEDIRS" (I'm still looking for a good
>> variable name), a recipe would be able to override it for the recipe as a whole,
>> or for the split packages.  Almost all recipes would not need to ever override
>> the value.
>>
>> ----
>>
>> The net result is any existing package will continue to work.
>>
>> A new distribution wide set of directories that regular recipes should not own
>> can be created.
>>
>> And the few key packages that need to own directories can do so, and be sure
>> that they are the exclusive owners.
>>
>> Comments, Suggestions?
>>
>> --Mark
>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core at lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
> 
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core