ltib
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Ltib] One step, CMO build


From: Peter Barada
Subject: Re: [Ltib] One step, CMO build
Date: Mon, 30 Sep 2013 14:12:37 -0400
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.0

On 09/30/2013 01:21 PM, Jump, Lance wrote:
Designation: Non-Finmeccanica
 
> >> Is it possible to archive (in SVN, for example) a complete package
> >> that allows LTIB to rebuild from “source” with a single command using
> >> the libraries that were archived at the time of archive?
> >>
> >> The situation is this. The developers will configure LTIB (kernel,
> >> RFS …) and download all packages needed. At some point, they will
> >> check this into an archive so that the Configuration Management
> >> Organization (CMO) can check it out and build it.
> >>
> >> The CMO build will have no access to the internet and the personnel,
> >> who are not familiar with LTIB, will want to invoke a script that
> >> does everything – even setting up /opt/freecale -- with no interaction.
> >>
> >> I have been able to do some this, such as setting up a local package
> >> pool and using preconfigure, but I cannot accomplish the whole thing
> >> without some manual interaction and/or retaining a significant amount
> >> of the build by- products.
> >>
> >> Ideally, I would like to save nothing more than the original source
> >> (LTIB and
> >> packages) and small number of configuration files. Then have a script
> >> that takes these and performs the complete build.
> >
> > Obviously you only want to save the LTIB source (and local package
> > pool with appropriate configuration in .ltibrc to point to it) in SVN
> > and have the CMO machine check it out and build "from scratch".  Of
> > course you have to assume that the CMO build machine has all the
> > necessary host packages to complete the build (i.e. build-essentials,
> > etc) as well as grants the "build user" appropriate sudo permissions.
>
> Yes, the CMO build machine would have to be configured. For now, I am
> experimenting using the development machine as the CMO machine and
> first running the build interactively there.
>
> >
> > Given that you could write a script that automates the process via:
> >
> > 1) check out latest version of the LTIB source directory (that
> > contains the local package pool that has copies of everything the
> > build pulls from the global package pool)
> > 2) rm -rf /opt/freescale
> > 3) rm -rf rpm
> > 4) sudo rm -rf rootfs*
> > 5) rm -f .host_warning*
> > 6) rm -f .tc_*
> > 7) ./ltib -b --preconfig config/platform/<platform>/<defconfig>
> >
> > Where "<platform>" is the platform directory and "<defconfig>" is the
> > default LTIB configuration file.
> > Note the ".tc_*" is the trigger file telling LTIB to skip installing
> > the toolchain package and ".host_warning*" is the trigger file telling
> > LTIB to skip building its host packages - hence removing them forces LTIB
> to repeat those steps.
> >
> > This should pretty much build LTIB completely from scratch...
> >
>
> The process I tried is similar to the above, except I also start with an empty
> working directory. So, steps 3 through 6 are implicit. I create the working
> directory by unpacking the original ltib skeleton.  At that point, everything is
> there (including the packages in the LPP) and I can run LTIB interactively.
 
< I use buildbot to manage continuous integration LTIB work for me, and I find
< I don't need to empty out the directory and completely check out a fresh
< tree - just the update is good enough, modulo removing the built files
<  mentioned above.
 
Okay, but in my case, I actually want to do a fresh checkout since it is to be built on a CMO machine, which, in general, will be different from the development machine.
 
> However, I do not see how to run it without user interaction. I have tried
> various combinations of "-b" and "--preconfig" with various saved
> configurations files. But I can't seem to get the magic set. Below are some
> that I have tried (ltib/ is a saved run and ../ltib is the fresh working copy):
>
> cp .ltibrc ../ltib
> cp ltib/config/main.lkc ../ltib/config/
> cp ltib/.config ../ltib/
> cp ltib/config/.config ../ltib/config/
> cp ltib/config/platform/imx/.config ../ltib/config/platform/imx/ cp
> ltib/config/platform/imx/defconfig.dev ../ltib/config/platform/imx/ cp
> ltib/config/platform/imx/imx6_defconfig.dev
> ../ltib/config/platform/imx/imx6_defconfig
>
> I am not sure which of these files I need, if I left out some crucial ones or
> included some that cause problems.
 
< I never use .dev files in my continuous integration builds - only the
< defconfigs - I have a step that does "rm -f
< config/platform/<platform>/*.dev" to remove them, and then run "./ltib -b
< --preconfig config/platform/<platform>/defconfig" to build it.  You
< shouldn't have to copy any of those files around to get it to work.
 
Well, remember that start with an "empty" LTIB working directory, which is just an untar of ltib.tar.gz as we received it. So, at that point, nothing has been configured or selected. I was hoping I could use some of the configuration files that were created when I ran it interactively on the development machine.
Sure, see below - there are very few if you are using a platform that exists in the tarball.  If you are creating a new platform then the list is a bit longer...
 
< When you run "./ltib -b --preconfig
< config/platform/<platform>/<defconfig>" from a _completely_ fresh
< checkout (with no other modifications to the tree), what user intervention
< is required?  You should fix that problem first - perhaps you've broken
< something in your configuration that LTIB forces you to go through the/a
< configuration step.
 
At this point, LTIB environment doesn't even know what processor or board I'm going to use. Unless I copy in some config files from the interractive, I don't see what file to use for a preconfig. If I run it without doing that, I get:
 
/opt/freescale/ltib/usr/bin/rpm has gone missing at ./ltib line 2245.
traceback:
main::legacy_rpm_fixups:2245
  main::pre_build_checks:1463
   main:569
 
But I'm not sure that is really a problem with LTIB since I am asking it to do something it can't have any way to know how. If I just run ltib with no parameters, it gives me menus to select the processor family and a bunch of other stuff as I would expect. Also, there shouldn't be any broken files because this is a completely fresh copy from the tar file as received from the site.
 
< If "-b --preconfig" takes you into the kernel config step, you'll want to edit
< your defconfig and disable CONFIG_PKG_KERNEL_WANT_CF since that's a
< temporary trigger that forces LTIB to run it.
 
I don't think there is a defconfig in a fresh ltib working directory -- or at least not one that I configured since I have not run LTIB yet.
Do "ls config/platform/*/defconfig" and you'll see the defconfigs...
 
The overall process I am trying to execute is this:
1. Starting with a fresh ltib.tar.gz file, do a manual/interactive build using LTIB.
2. Save all packages downloaded during the process.
3. Save any necessary configuration files that describe what has been selected.
4. Deliver ltib.tar.gz and the packages and configuration from steps 2 and 3 to CMO.
5. At the CMO machine take the ltib, packages and configuration files and run a script to create the environment that resulted at the end of step 1.
 
The key ideas here are:
a) very few byproducts of the original build (step 1) are saved;
b) the LTIB working and opt directories are completely erased; and
c) the CMO agent need only run a script with no interaction -- especially any that would require domain/project knowledge.

1) The only files you need to save are:

- The packages downloaded into /opt/ltib/pkgs (they need to be copied into your local package pool)

- The platform defconfig.dev file (LTIB saves changes to the platform defconfig in defconfig.dev).  To save do "mv config/platform/<platform>/defconfig.dev config/platform/<platform>/defconfig" (don't use cp as the defconfig.dev takes precidence) after you're happy with the build after you've modified the configuration

- The kernel config .dev file (if you changed the kernel configuration).  Again mv it into its appropriate place (can't tell you exactly since each platform/kernel combination uses different names for the kernel defconfig).

2) The only command the CMO should need to run to recreate your build "./ltib -b --preconfig config/platform/<platform>/defconfig"

That's it.

As an experiment, pull out the fresh LTIB tarball, unpack it and check it into SVN.  Then check it out and do your build.  Use "svn status" to find out anything that changed and any newly created files.  mv your selected platform defconfig back to defconfig (in the same directory).  Then remove _anything_ else that was freshly generated (i.e "sudo rm -rf "svn status | grep '^?' | awk '{print $2}'").  Run "./ltib -b --preconfig config/platform/<platform>/defconfig" and it should do what you want...
-- 
Peter Barada
address@hidden

reply via email to

[Prev in Thread] Current Thread [Next in Thread]