[oe] [meta-networking][PATCH] GeoIP: change data fetching strategy

[Martin Jansa]

On Wed, Jan 07, 2015 at 07:41:06PM -0800, Khem Raj wrote:
> 
> > On Jan 7, 2015, at 6:19 PM, Joe MacDonald <joe_macdonald at mentor.com> wrote:
> > 
> > Since the GeoIP databases are not versioned and they essentially change
> > without notice, we cannot reasonably expect the checksums to be consistent
> 
> isn’t this like AUTOREV ?. IMO such packages should be mirrored somewhere
> regularly and used. if you maintain this recipe then please mirror the sources
> somewhere and then update them regularly, but from one build to another is just
> too frequent.

Agreed, we should also make sure that once PREMIRROR is fully populated
we can build without any access to network. This is breaking that
assumption and doesn't even use PREMIRROR to "cache" them.

> > from one build to the next.  Therefore, don't download them using the
> > usual wget fetcher, but do it manually at the end of the do_fetch stage
> > and place the uncompressed data files where the remainder of the build
> > expects they would've been placed.
> > 
> > Signed-off-by: Joe MacDonald <joe_macdonald at mentor.com>
> > ---
> > 
> > I considered trying to use the wget fetcher and having it skip over the
> > checksum operation and, honestly, I'd still prefer to do it that way, but
> > looking around and at the fetch2 code, it doesn't appear to be able to support
> > that outside of a BB_STRICT_CHECKSUM="" setting, which we definitely don't want
> > to do here.
> > 
> > -J.
> > 
> > .../recipes-support/geoip/geoip_1.6.0.bb           | 36 +++++++++-------------
> > 1 file changed, 14 insertions(+), 22 deletions(-)
> > 
> > diff --git a/meta-networking/recipes-support/geoip/geoip_1.6.0.bb b/meta-networking/recipes-support/geoip/geoip_1.6.0.bb
> > index 65e8362..71fda3b 100644
> > --- a/meta-networking/recipes-support/geoip/geoip_1.6.0.bb
> > +++ b/meta-networking/recipes-support/geoip/geoip_1.6.0.bb
> > @@ -8,28 +8,9 @@ using reverse DNS lookups."
> > HOMEPAGE = "http://dev.maxmind.com/geoip/"
> > SECTION = "Development/Libraries"
> > 
> > -SRC_URI = "http://www.maxmind.com/download/geoip/api/c/GeoIP-1.6.0.tar.gz;name=tarball \
> > -           http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz;apply=no;name=GeoIP-dat \
> > -           http://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz;apply=no;name=GeoIPv6-dat \
> > -           http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz;apply=no;name=GeoLiteCity-dat \
> > -           http://geolite.maxmind.com/download/geoip/database/GeoLiteCityv6-beta/GeoLiteCityv6.dat.gz;apply=no;name=GeoLiteCityv6-dat \
> > -"
> > -
> > -SRC_URI[tarball.md5sum] = "89f4cdfdab43f1d67364cd7c85bbe8ca"
> > -SRC_URI[tarball.sha256sum] = "075a0c2815cd099e9ec35c9569db716a3fefcdbb6a10dbfa1ce7c6cd48d4a635"
> > -
> > -SRC_URI[GeoIP-dat.md5sum] = "4bc1e8280fe2db0adc3fe48663b8926e"
> > -SRC_URI[GeoIP-dat.sha256sum] = "7fd7e4829aaaae2677a7975eeecd170134195e5b7e6fc7d30bf3caf34db41bcd"
> > -
> > -SRC_URI[GeoIPv6-dat.md5sum] = "aac7e6e9b141de80934ecee52daf7f56"
> > -SRC_URI[GeoIPv6-dat.sha256sum] = "126fd2953eb193e60538e30b4465610530383f7a782745cacdca5ba6825f471c"
> > -
> > -SRC_URI[GeoLiteCity-dat.md5sum] = "15a42c684c53d2309e6632a6d6e02531"
> > -SRC_URI[GeoLiteCity-dat.sha256sum] = "5ec02a6d39d545c77ec12cc30c6a8856883d8f55522fc5cd4f25af80163c6b3c"
> > -
> > -SRC_URI[GeoLiteCityv6-dat.md5sum] = "49d6ec946fa0a2575b5112a68d71f933"
> > -SRC_URI[GeoLiteCityv6-dat.sha256sum] = "7a345e6cf0e59f8ab589ff15020241f0b03342dd04cc584f814c4f4700d49405"
> > -
> > +SRC_URI = "http://www.maxmind.com/download/geoip/api/c/GeoIP-1.6.0.tar.gz"
> > +SRC_URI[md5sum] = "89f4cdfdab43f1d67364cd7c85bbe8ca"
> > +SRC_URI[sha256sum] = "075a0c2815cd099e9ec35c9569db716a3fefcdbb6a10dbfa1ce7c6cd48d4a635"
> > 
> > LICENSE = "LGPL-2.1"
> > 
> > @@ -43,6 +24,17 @@ inherit autotools
> > EXTRA_OECONF = "--disable-static               \
> >                 --disable-dependency-tracking  "
> > 
> > +do_fetch_append() {
> > +    os.system("${FETCHCMD_wget} http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz");
> > +    os.system("gunzip -c ${DL_DIR}/GeoIP.dat.gz > ${WORKDIR}/GeoIP.dat");
> > +    os.system("${FETCHCMD_wget} http://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz");
> > +    os.system("gunzip -c ${DL_DIR}/GeoIPv6.dat.gz > ${WORKDIR}/GeoIPv6.dat");
> > +    os.system("${FETCHCMD_wget} http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz");
> > +    os.system("gunzip -c ${DL_DIR}/GeoLiteCity.dat.gz > ${WORKDIR}/GeoLiteCity.dat");
> > +    os.system("${FETCHCMD_wget} http://geolite.maxmind.com/download/geoip/database/GeoLiteCityv6.dat.gz");
> > +    os.system("gunzip -c ${DL_DIR}/GeoLiteCityv6.dat.gz > ${WORKDIR}/GeoLiteCityv6.dat");
> > +}
> > +
> > do_install() {
> >     make DESTDIR=${D} install
> >     install -d ${D}/${datadir}/GeoIP
> > -- 
> > 1.9.1
> > 
> > -- 
> > _______________________________________________
> > Openembedded-devel mailing list
> > Openembedded-devel at lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-devel
> 
> -- 
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-devel

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.openembedded.org/pipermail/openembedded-devel/attachments/20150108/13bc5776/attachment-0002.sig>