Re: How to install all toolboxes with one command?

[Sergei Steshenko]

----- Original Message -----
> From: Carnë Draug <address@hidden>
> To: Nash A <address@hidden>
> Cc: Jordi Gutiérrez Hermoso <address@hidden>; "address@hidden" 
> <address@hidden>
> Sent: Thursday, June 28, 2012 3:27 PM
> Subject: Re: How to install all toolboxes with one command?
> 
> On 28 June 2012 03:05, Jordi Gutiérrez Hermoso <address@hidden> wrote:
>>  On 27 June 2012 19:32, Nash A <address@hidden> wrote:
>>>  I am new to octave. I searched but not able to find such a command.
>>> 
>>>  I am on linux. I'd like to install all available octave packages in 
> one
>>>  command.
>> 
>>  This isn't desirable. It's not even possible. Some packages 
> conflict
>>  with each other. Some are broken. Others are just crappy and will
>>  break your system.
> 
> I will subscribe to Jordi's words.
> 
> Also, this has been asked enough times to deserve an entry on the
> mailing list. Please see
> http://octave.org/wiki/index.php?title=FAQ#How_do_I_install_all_octave_packages.3F
> 
> Carnë
> _______________________________________________


It _is_ desirable (with some limitations). It's much easier to install 
everything and then to uninstall the unneeded/undesirable packages.

In fact, this is what I am routinely doing in my AppsFromScratch builds.

And the packages I consider to be bad from the getgo are:

          my %packages_to_exclude =
            (
            bugfix => '',
            java => '',
            plot => '',
            windows => '',
            oct2mat => ''
            );
.

Regards,
  Sergei.

P.S. The piece of code installing all the packages in my AppsFromScratch - pay 
attentions to

# install packages using a number of iterations in order to satisfy dependencies

part:

      octave_forge_bundle =>
        do{
          my $versioned_target = 'octave_forge_bundle';
          my $global_or_local = 'local';

          my %packages_to_exclude =
            (
            bugfix => '',
            java => '',
            plot => '',
            windows => '',
            oct2mat => ''
            );

          my $octave_versioned_target =
          $gen_versioned_target_name_sub->
            (
            \%shadow_targets_hash_refs,
            'octave'
            );

          my $octave_install_prefix =
          exists $shadow_targets_hash_refs{octave}{install_prefix}
          ? $shadow_targets_hash_refs{octave}{install_prefix}
          : "${$cmdline_args_hash_ref}{install_prefix}/$octave_versioned_target"
          ;
          my $octave_exec = "$octave_install_prefix/binsh/octave";

          my $tarball_to_package_sub =
          sub
            {
            my ($tarball) = @_;
            (my $package) = ($tarball =~ m|^(.*?)-\d.*\.tar.gz$|);
            $package;
            };

          $shadow_targets_hash_refs{$versioned_target} =
            {
            dont_cleanup_install_dir => 1,

            base_urls => [],

            get_source_tree_sub =>
              sub
                {
                my %this_sub_args = @_;
                
                $system_wrapper_sub->($info_marker, "\\mkdir -p 
$this_sub_args{target_build_dir}");
                return 1 unless -d $this_sub_args{target_build_dir};

                my $tarball = " $this_sub_args{versioned_target}.tar.bz2";

                return 1 if($system_wrapper_sub->($info_marker, "\\cd 
$this_sub_args{target_build_dir}/.. && tar jcf $tarball 
$this_sub_args{versioned_target}") != 0);

                0;
                },


            pre_configure_sub =>
              sub
                {
                my %this_sub_args = @_;

                my $install_prefix = 
"${$cmdline_args_hash_ref}{install_prefix}/$versioned_target";

                #my $already_built_file = 
"$install_prefix/___ALREADY_BUILT___.$versioned_target "; warn 
"\$already_built_file=$already_built_file";
                #return 0 if -e $already_built_file;

                my $target_build_dir = 
"${$cmdline_args_hash_ref}{build_prefix}/$versioned_target";

                return 1 if $system_wrapper_sub->($info_marker, "\\mkdir -p 
$target_build_dir");

                $system_wrapper_sub->($info_marker, "\\rm -rf $install_prefix");
                return 1 if $system_wrapper_sub->($info_marker, "\\mkdir -p 
$install_prefix");

                my $packages_front_page = 
'http://octave.sourceforge.net/packages.php';
                my $saved_packages_front_page = 
"$target_build_dir/packages.html";
                return 1 if $system_wrapper_sub->($info_marker, "\\wget -nv -t0 
$packages_front_page -O $saved_packages_front_page");

                my @tarballs;

                # download packages
                  {
                  open(my $fh, '<', $saved_packages_front_page) or do{warn 
"$error_marker cannot open '$saved_packages_front_page' file for reading\n"; 
return 1};

                  my %u;
                  while(defined(my $line = <$fh>))
                    {
                    if($line =~ 
m|\"(http\://downloads\.sourceforge\.net/octave/)(.*?)\?download\"|) # "
                      {
                      #warn "$line=$line";
                      my $url_up_to_tarball = $1;
                      my $tarball = $2; # warn "\$tarball=$tarball";

                      my $package = $tarball_to_package_sub->($tarball); #warn 
"\$package=$package";
                      unless(defined $package)
                        {
                        warn "$warning_marker \$tarball=$tarball does not 
follow foo-bar.tar.gz naming convention, skipping it\n";
                        next;
                        }

                      next if exists $packages_to_exclude{$package};

                      my $full_path_tarball = "$target_build_dir/../$tarball";

                      unless(-e $full_path_tarball)
                        {
                        if($system_wrapper_sub->($info_marker, "\\cd 
$target_build_dir/../ && \\wget -nv -t0 -c 
http://downloads.sourceforge.net/octave/$tarball";) != 0)
                          {
                          warn "$error_marker could not download '$tarball' 
file\n";
                          next;
                          }
                        }

                      

                      unless(exists $u{$tarball})
                        {
                        if($system_wrapper_sub->($info_marker, "\\cd 
$target_build_dir && \\ln -s -f ../$tarball .") != 0)
                          {
                          warn "$error_marker could not link '$tarball' file\n";
                          return 1;
                          }

                        push @tarballs, $tarball;
                        }

                      $u{$tarball} = '';
                      }
                    }

                  close($fh);

                  @tarballs = sort(@tarballs);
                  }

                # uninstall old packages if any
                  {
                  foreach my $tarball(@tarballs)
                    {
                    my $package = $tarball_to_package_sub->($tarball);

                    unless(defined $package)
                      {
                      warn "$warning_marker \$tarball=$tarball does not follow 
NAME-VER.tar.gz naming convention, skipping it\n";
                      next;
                      }

                    next if exists $packages_to_exclude{$package};

                    my $log_file = "$package.uninstall.log";

                    $system_wrapper_sub->($info_marker, "\\cd $target_build_dir 
&& $octave_exec -V --no-init-file --eval 'pkg uninstall -verbose -nodeps 
-$global_or_local $package' 1>$log_file 2>&1");
                    } # foreach my $tarball(@tarballs)
                  }

                # create /bin/sh wrapper scripts for packages installation
                  {
                  foreach my $tarball(@tarballs)
                    {
                    my $sh_script = "$target_build_dir/$tarball.sh";
                    open(my $sh_script_fh, '>', $sh_script) or
                    do{
                      warn "$error_marker could not open '$sh_script' file for 
writing\n";
                      return 1;
                      };

                    print $sh_script_fh <<EOD
TMP=$target_build_dir
export TMP

LD_LIBRARY_PATH='$this_sub_args{LD_LIBRARY_PATH}'
export LD_LIBRARY_PATH

LDFLAGS='$this_sub_args{LDFLAGS}'
export LDFLAGS

CPPFLAGS='$this_sub_args{CPPFLAGS}'
export CPPFLAGS

PATH='$this_sub_args{PATH}'
export PATH

PKG_CONFIG_PATH='$this_sub_args{PKG_CONFIG_PATH}'
export PKG_CONFIG_PATH

$octave_exec -V -H --no-init-file --eval 'pkg install -$global_or_local 
-verbose -auto $tarball'
EOD
                    ;
                    close($sh_script_fh);
                    }
                  }

#                # patch certain packages
#                  {
#                  my $octave_versioned_target =
#                  $gen_versioned_target_name_sub->
#                    (
#                    \%shadow_targets_hash_refs,
#                    'octave'
#                    );
#
#                    {
#                    my $package = 'ocs';
#                    #my $version = '0.1.1';
#            my $version = '0.1.3';
#            
#                    my $versioned_package = "$package-$version";
#                    my $tarball = "$versioned_package.tar.gz";
#
#                    return 1 unless($system_wrapper_sub->($info_marker, "\\cd 
$target_build_dir && \\tar zxvf $tarball") == 0);
#
#                    foreach my $file(qw(PKG_ADD PKG_DEL))
#                      {
#                      return 1 unless($system_wrapper_sub->($info_marker, 
"\\cp -p 
$tool_path/../include/perl/project_specific/$octave_versioned_target/octave_forge_bundle/$versioned_package/$file
 $target_build_dir/$package/") == 0);
#                      }
#
#                    return 1 unless($system_wrapper_sub->($info_marker, "\\cd 
$target_build_dir && \\tar zcvf $tarball $package") == 0);
#                    }
#                  }

                # install packages using a number of iterations in order to 
satisfy dependencies
                  {
                  my $installed_packages_file = 
"$target_build_dir/___installed_packages___.txt";
                  my $query_installed_packages_log = 
"$target_build_dir/query_installed_packages.log";

                  my %already_installed_packages;
                  my $number_of_already_installed_packages = 0;
                  for(my $iter = 0; 1; $iter++)
                    {
                    warn "$info_marker $versioned_target: \$iter=$iter BEGIN\n";

                    foreach my $tarball(@tarballs)
                      {
                      my $package = $tarball_to_package_sub->($tarball);

                      unless(defined $package)
                        {
                        next;
                        }

                      next if exists $packages_to_exclude{$package};
                      next if exists $already_installed_packages{$package};

                      my $log_file = "$tarball.install.log";
                      my $sh_script = "$target_build_dir/$tarball.sh";

                      $system_wrapper_sub->($info_marker, "\\cd 
$target_build_dir && /bin/sh $sh_script 1>$log_file 2>&1");
                      }

                    unlink($installed_packages_file);
                    if(-e $installed_packages_file)
                      {
                      warn "$error_marker could not delete 
'$installed_packages_file' file\n";
                      return 1
                      }

                    $system_wrapper_sub->($info_marker, "$octave_exec -H -q 
--no-init-file --eval 
'p=pkg(\"list\");f=fopen(\"$installed_packages_file\",\"w\");for k=1:numel(p) 
fprintf(f,\"%s\\n\",p{k}.name);endfor;fclose(f)' 
1>$query_installed_packages_log 2>&1");

                    open(my $fh, '<', $installed_packages_file) or do{warn 
"$error_marker can not open '$installed_packages_file' file for reading\n"; 
return 1};

                    while(defined(my $package = <$fh>))
                      {
                      chomp($package);
                      $already_installed_packages{$package} = '';
                      }

                    close($fh);

                    warn "$info_marker $versioned_target: \$iter=$iter END\n\n";

                    if($number_of_already_installed_packages == scalar(keys 
%already_installed_packages))
                      {
                      warn "$info_marker no increase of 
\$number_of_already_installed_packages=$number_of_already_installed_packages, 
abandoning the loop\n";
                      last;
                      }

                    $number_of_already_installed_packages = scalar(keys 
%already_installed_packages);
                    warn "$info_marker 
\$number_of_already_installed_packages=$number_of_already_installed_packages\n";
                    }
                  }

                #return 1 unless open(my $fh, '>', $already_built_file);
                #close($fh);


                return if $_create_dummy_configure_sub->(%this_sub_args);
                return if $_create_dummy_Makefile_sub->(%this_sub_args);

                0;
                },

            make_targets => [''],

            post_sub =>
              sub
                {
                $gen_versioned_target_name_sub->
                  (
                  \%shadow_targets_hash_refs,
                  'octave'
                  );
                my $octave_install_prefix =
                exists $shadow_targets_hash_refs{octave}{install_prefix}
                ? $shadow_targets_hash_refs{octave}{install_prefix}
                : 
"${$cmdline_args_hash_ref}{install_prefix}/$octave_versioned_target"
                ;
                $system_wrapper_sub->($info_marker, "\\chmod -R a+r 
$octave_install_prefix");
                0;
                },

            depends_on =>
              {
              targets =>
                {
                octave => $shadow_targets_hash_refs{octave}
                }
              }
            }
          },