<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p style="margin-top:0;margin-bottom:0"></p>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
Hello Alejandro,</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
Thank you for your help.<br>
<br>
Unfortunately, the pyvenv package is not in the manifest, I cannot therefore add the path where you are suggesting.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
It is not immediately clear to me where the package is created, however, files are added to the pyvenv package (using FILES_${PN}-pyvenv += ) in the python3_3.5.6.bb recipe. This is why I added the ${libdir}/${PYTHON_MAJMIN}/venv file in a bbapend to the
<span style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px;">python3_3.5.6.bb</span> recipe.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
I am able to successfully run the create_manifest3.py script but the generated manifest content has not changed. I did not expect it to change because pyvenv was not there in the first place.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
On a different note; I managed to include the venv module by including the python3-misc package to the build. I can create a virtual environment using python -m venv. The issue with that is that python3-misc includes other modules I do not need and thus needlessly
increases size. Another issue is that I have to include python3-runpy (to use the -m option when creating the virtual environment).</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
I also realised that python3-pip is a depencency to venv.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
In conclusion, can you think of how dependencies (venv and pip) can be added to the python3-pyvenv package? This will allow reduction of the image size by removing python3-misc and python3-runpy.<br>
<br>
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 16px; margin-top: 0px; margin-bottom: 0px;">
Hugues</div>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Alejandro Enedino Hernandez Samaniego <aehs29@gmail.com><br>
<b>Sent:</b> 19 October 2018 13:37:14<br>
<b>To:</b> Hugues Kamba<br>
<b>Cc:</b> Burton, Ross; Jonathan Haigh; OE-core; Jeremy Johnson<br>
<b>Subject:</b> Re: [OE-core] Missing venv module in Python 3</font>
<div> </div>
</div>
<meta content="text/html; charset=utf-8">
<div>
<div dir="auto">
<div>Hello Huges,
<div dir="auto"><br>
</div>
<div dir="auto">The path needs to be added to the manifest itself (the json file) under the files section for the pyvenv package not in a bbappend.</div>
<div dir="auto"><br>
</div>
<div dir="auto">And to run the create manifest task you would do:</div>
<div dir="auto"><br>
</div>
<div dir="auto">$ bitbake python3 -c create_manifest</div>
<div dir="auto"><br>
</div>
<div dir="auto">Once I get to my computer I will be able to provide more help.</div>
<div dir="auto"><br>
</div>
<div dir="auto">Alejandro</div>
<br>
<br>
<div class="x_gmail_quote">
<div dir="ltr">On Fri, Oct 19, 2018, 10:08 AM Hugues Kamba <<a href="mailto:Hugues.Kamba@arm.com">Hugues.Kamba@arm.com</a>> wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div id="x_m_-4307242440526293090divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p style="margin-top:0; margin-bottom:0">Hi Alejandro,</p>
<p style="margin-top:0; margin-bottom:0"><br>
</p>
<p style="margin-top:0; margin-bottom:0">Thank you for your suggestion.</p>
<p style="margin-top:0; margin-bottom:0"><br>
</p>
<p style="margin-top:0; margin-bottom:0">The following has been added to a python3_%.bbapend:</p>
<p style="margin-top:0; margin-bottom:0"></p>
<div><i><span style="font-size:10pt"># Add missing venv module dependency.</span><br>
<span style="font-size:10pt">FILES_${PN}-pyvenv += "${libdir}/${PYTHON_MAJMIN}/venv"</span></i></div>
<br>
<p></p>
<p style="margin-top:0; margin-bottom:0">However, it is not quite obvious how to successfully run the
<i><span style="font-size:10pt">create_manifest3.py</span></i> script.<br>
Attempts were made to run it with "<span style="font-size:10pt">python3 create_manifest3.py python3.5"</span> but it complains about an undefined variable (<i><span style="font-size:10pt">nativelibfolder</span></i>).<br>
I also tried to run it using "<i><span style="font-size:10pt">bitbake pyhton -c create_manifest3.py</span>"</i> as specified in the docstring but execution fails because of missing tasks (no do_..).</p>
<p style="margin-top:0; margin-bottom:0"><br>
</p>
<p style="margin-top:0; margin-bottom:0">Could you or others on the mailing list, provide some guidance on how to go about running (arguments, etc..) the
<i><span style="font-size:10pt">create_manifest3.py</span></i> script. </p>
<div id="x_m_-4307242440526293090Signature">
<div id="x_m_-4307242440526293090divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p style="margin-top:0; margin-bottom:0"><br>
</p>
<table align="left" border="0" cellspacing="0" cellpadding="0" style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:15px; border-collapse:collapse; margin-right:6.75pt; margin-left:6.75pt">
<tbody>
<tr>
<td width="147" style="width:88.25pt; padding:0px 5.4pt; border-style:none solid none none; border-right-width:1pt; border-right-color:windowtext">
<div style="margin:0px"><img class="x_m_-4307242440526293090EmojiInsert" data-outlook-trace="F:0|T:1" src="cid:bbaed11e-d9b2-456f-bd15-200669ad237e"><br>
</div>
</td>
<td width="419" style="width:251.95pt; vertical-align:top; padding:0px 5.4pt">
<div style="margin:0px"><font face="Calibri,sans-serif" size="2"><span style="font-size:11pt"><font size="2" color="black"><span lang="it" style="font-size:10pt"><b>Hugues Kamba </b></span></font><font size="2" color="black"><span lang="it" style="font-size:10pt">|
Senior Software Engineer</span></font></span></font></div>
<div style="margin:0px"><font face="Calibri,sans-serif" size="2"><span style="font-size:11pt"><a href="mailto:sandi.shannon@arm.com" rel="noopener noreferrer noreferrer" id="x_m_-4307242440526293090LPNoLP" title="mailto:sandi.shannon@arm.com
Ctrl+Click or tap to follow the link" target="_blank"><font color="#0563C1"><span lang="it">hugues.kamba</span></font><font size="2" color="#0563C1"><span lang="it" style="font-size:10pt">@arm.com</span></font></a></span></font></div>
<div style="margin:0px"><font face="Calibri,sans-serif" size="2"><span style="font-size:11pt"><font size="2" color="black"><span style="font-size:10pt"><span style="color:rgb(68,84,106); font-family:Calibri,sans-serif; font-size:12px">CPC1, Capital Park, Cambridge Road, Fulbourn, CB21
5XE, UK</span></span></font></span></font></div>
<span style="color:rgb(33,33,33); font-family:Calibri,sans-serif; font-size:12px">Tel: (+44)1223 400400</span>
<div style="margin:0px"><font face="Calibri,sans-serif" size="2"><span style="font-size:11pt"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__community.arm.com_welcome&d=DwMFAg&c=bKNseOHsszsRCtLFfd6XtA&r=h1JGsEgvfYBPUFak8b6Fs-oV6irVsuuFc4RBESlsTFE&m=N8x4mnijl-wwLmtz3tbzVbe9R-QRW9bVZHfjOv3YY0o&s=FTtAextvaqUli74ujYjbDxp7jK9aaA6G7NOsVfryuyw&e=" rel="noopener noreferrer noreferrer" id="x_m_-4307242440526293090LPNoLP" target="_blank"><font size="2"><span style="font-size:10pt"><font color="blue">Visit
the new ARM Connected Community</font></span></font></a></span></font></div>
</td>
</tr>
</tbody>
</table>
<p></p>
</div>
</div>
<br>
<br>
<div style="color:rgb(0,0,0)">
<hr style="display:inline-block; width:98%">
<div id="x_m_-4307242440526293090divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Alejandro Enedino Hernandez Samaniego <<a href="mailto:aehs29@gmail.com" target="_blank" rel="noreferrer">aehs29@gmail.com</a>><br>
<b>Sent:</b> 18 October 2018 19:18<br>
<b>To:</b> Burton, Ross<br>
<b>Cc:</b> Jonathan Haigh; OE-core; Jeremy Johnson; Hugues Kamba<br>
<b>Subject:</b> Re: [OE-core] Missing venv module in Python 3</font>
<div> </div>
</div>
<div>
<div dir="auto">
<div>Hello folks,<br>
<br>
<div class="x_m_-4307242440526293090x_gmail_quote">
<div dir="ltr">On Thu, Oct 18, 2018, 3:34 PM Burton, Ross <<a href="mailto:ross.burton@intel.com" target="_blank" rel="noreferrer">ross.burton@intel.com</a>> wrote:<br>
</div>
<blockquote class="x_m_-4307242440526293090x_gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
On Thu, 18 Oct 2018 at 16:19, Jonathan Haigh <<a href="mailto:Jonathan.Haigh@arm.com" rel="noreferrer noreferrer" target="_blank">Jonathan.Haigh@arm.com</a>> wrote:<br>
> The venv package (<a href="https://docs.python.org/3/library/venv.html" rel="noreferrer noreferrer noreferrer" target="_blank">https://docs.python.org/3/library/venv.html</a>) itself is missing although its source is available within Python 3.5 (<a href="https://www.python.org/ftp/python/3.5.6/Python-3.5.6.tgz" rel="noreferrer noreferrer noreferrer" target="_blank">https://www.python.org/ftp/python/3.5.6/Python-3.5.6.tgz</a>).
As mentioned earlier, it seems the issue is because there is no entry in the manifest from which the various Python 3 modules are created (<a href="https://github.com/openembedded/openembedded-core/blob/master/meta/recipes-devtools/python/python3/python3-manifest.json" rel="noreferrer noreferrer noreferrer" target="_blank">https://github.com/openembedded/openembedded-core/blob/master/meta/recipes-devtools/python/python3/python3-manifest.json</a>).<br>
><br>
> The manifest is generated from <a href="https://github.com/openembedded/openembedded-core/blob/master/meta/recipes-devtools/python/python3/create_manifest3.py" rel="noreferrer noreferrer noreferrer" target="_blank">
https://github.com/openembedded/openembedded-core/blob/master/meta/recipes-devtools/python/python3/create_manifest3.py</a>, and it does not seem obvious to me how to go about correctly adding a new module to the manifest, using create_manifest3.py or otherwise.<br>
</blockquote>
</div>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">The script itself should get the venv dependency automatically (my guess is that this isn't getting detected by the script because the module itself its coming from a directory) but I'd have to debug what is exactly happening, although I don't
have access to my computer at the moment.</div>
<div dir="auto"><br>
</div>
<div dir="auto">An easy fix (for now) is to add the</div>
<div dir="auto"><br>
</div>
<div dir="auto"><span style="font-family:sans-serif">/usr/lib/python3.5/venv/</span><br>
</div>
<div dir="auto"><span style="font-family:sans-serif"><br>
</span></div>
<div dir="auto"><span style="font-family:sans-serif">Path to the manifest on the FILES for the pyenv module.</span></div>
<div dir="auto"><span style="font-family:sans-serif"><br>
</span></div>
<div dir="auto"><span style="font-family:sans-serif">After that running the create_manifest task should get its dependencies correctly and update the manifest so it can be pushed to the repo.</span></div>
<div dir="auto"><span style="font-family:sans-serif"><br>
</span></div>
<div dir="auto"><span style="font-family:sans-serif">Cheers,</span></div>
<div dir="auto"><span style="font-family:sans-serif"><br>
</span></div>
<div dir="auto"><span style="font-family:sans-serif">Alejandro</span></div>
<div dir="auto"><span style="font-family:sans-serif"><br>
</span></div>
<div dir="auto"><span style="font-family:sans-serif"><br>
</span></div>
<div dir="auto">
<div class="x_m_-4307242440526293090x_gmail_quote">
<blockquote class="x_m_-4307242440526293090x_gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<br>
Ah, right.<br>
<br>
In that case:<br>
<br>
$ oe-pkgdata-util find-path */*venv*py<br>
python3-tests: /usr/lib/python3.5/test/test_venv.py<br>
python3-misc: /usr/lib/python3.5/venv/__main__.py<br>
python3-misc: /usr/lib/python3.5/venv/__init__.py<br>
<br>
The files are in the python3-misc package, which is the fallback for<br>
everything which isn't explicitly put into another package. I'd say<br>
the best fix for this would be to edit the manifest so the venv module<br>
goes into the python3-pyvenv package too.<br>
<br>
Ross<br>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose,
or store or copy the information in any medium. Thank you. </div>
</blockquote>
</div>
</div>
</div>
</div>
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose,
or store or copy the information in any medium. Thank you.
</body>
</html>