[OE-core] [PATCH 0/1] genext2fs: support large files and filesystems without using large amounts of memor

[Cui, Dexuan]

Darren Hart wrote on 2012-03-30:
> On 03/29/2012 03:07 PM, Richard Purdie wrote:
>> On Thu, 2012-03-29 at 00:51 +0800, Dexuan Cui wrote:
>>> Hi RP, Saul, Paul, Darren, Mark, Josh and joaohf and all, please comment.
>>> 
>>> Let's figure out if this big patch is acceptable or not...
>>> 
>>> With this patch, I can successfully create a 8.5GB .ext3 file with genext2fs.
>>> The speed is slow -- I spent about 1.5 hours.
>> 
>> If these patches solved all the problems and made things work
>> wonderfully I'd probably say we'd take them. Unfortunately I don't
>> consider taking 1.5 hours to build am 8GB filesystem "wonderful",
>> its rather worrying and I don't think its performing any where need
>> fast enough for our needs :(.
>> 
>> Patching genext2fs at this point in the cycle is a rather risky
>> undertaking too and I'm getting very concerned about this.
I agree. 
I understand the risk.

But, BTW, I personally tend to think the quality of the patches (which
Come from the genext2fs mailing list) are good:
please see my test results
http://sourceforge.net/mailarchive/message.php?msg_id=29062014

>> We might have to look at alternative ways to solve this problem. I
>> think Darren said he might help look at this and has some other ideas.
> Darren?
> 
> Without having looked into this very far, I seem to recall this was
> generating sparse files. This means every time we copy something into
> the filesystem it has to allocate the space on the drive. This is
> great for building big filesystems that will be mostly empty - but for
> big filesystems that we plan to mostly populate, I believe we would be
> better off doing pre-allocation (I'm taking this from my experience
> creating VMs in virtmanager). However, since using dd to create the
> file would require mounting it in order to copy files across (and we
> want to avoid requiring root permissions during build), this may not be an option.
> 
> Have we tried without the -z option? (-z allows holes in files - I
> presume this means sparse files).
I tried the -z option of genext2fs, but the speed to create the filesystem
is the same.

> 
> Depending on how much of the 8.5 GB image we are filling, we may get a
> significant speed increase by first generating a smaller image and then
> using resize2fs to grow it to the desired size (referencing
> http://landley.livejournal.com/47024.html).
> 
> 
> A quick test without passing an initial rootfs showed no difference
> between -z and no -z. It may still be worth trying with the actual
> population to see if that makes a difference.
>

Thanks,
-- Dexuan