.. _Pickled-Bundles:

Proper Bundles
==============

Prior to 2024, bundles were constructed for the purpose of solves, but
all other processing (e.g., computing W values) was done on individual
scenarios. These were called `loose bundles`.

.. Warning::
   Loose bundles (``--bundles-per-rank``) were removed in 2026.
   Setting ``bundles_per_rank > 0`` now raises a ``RuntimeError``.
   Use proper bundles (``--scenarios-per-bundle``) instead.

In 2024, `proper bundles` were supported. After the extensive form
for a proper bundle is created, the original scenarios are more or less
forgotten and all processing takes place for the bundle. At the time
of this writing, these bundles are a little less flexible in that
the number of scenarios per bundle must divide the number of scenarios
and randomizing the assignment of scenarios to bundles is left to the
user (e.g., by using a pseudo-random vector to provide one level
of indirection for the scenario number in the ``scenario_creator`` function).
As of the time of this writing, proper bundles always result in two-stage problems once the
bundles are created, even for problems that are multi-stage before bundling.
Proper bundles result in faster execution.

See ``mpisppy.generic_cylinders.py`` for an example of their use in
code and see ``examples.generic_cylinders.bash`` for a few proper
bundle command lines. Bundles can be created on the fly and used in
the same run, or they can be pickled to disk and reused on subsequent
runs. The ``--scenarios-per-bundle`` option must be specified for
every run that uses bundles (including runs that read pickled bundles).

Pickling and unpickling proper bundles — including pre-pickle
preprocessing (presolve, user callback, iter0 solve) and the tuning
workflow — is documented in :ref:`pickling`.

.. Note::
   If you do pseudo random number generation on-the-fly during scenario creation,
   very careful management of random seeds is required if you want to
   get the same scenarios with proper  bundles that you get without them.


Modules
-------

In addition to command line options specified in ``mpisppy.utils.config.py``
and supported in ``mpisppy.generic_cylinders.py``,
there are two modules that have most of the support for proper bundles:

- ``mpisppy.utils.pickle_bundle.py`` has miscellaneous utilities related to picking and other data processing
- ``mpisppy.utils.proper_bundler.py`` has wrappers for cylinder programs


Multistage
----------

There is support for multi-stage bundles in ``generic_cylinders.py``,
but the bundles must span the same number of entire second stage
nodes.

The most flexible way to create proper bundles is to write
your own problem-specific code to do it. The
file ``aircond_cylinders.py`` in the aircond example directory
provides an example.  The latter part of the ``allways.bash`` script
demonstrates how to run it.


Notes
-----

Pickled bundles are clearly useful for algorithm tuning and algorithm
experimentation. In some, but not all, settings they can also improve
wall-clock performance for a single optimization run. The pickler
(e.g., ``bundle_pickler.py`` in the aircond example) does not use a
solver and can be run once to provide bundles to all cylinders. It can
often be assigned as many ranks as the total number of CPUs
available. Reading the bundles from a pickle file is much faster
than creating them.

The trick is that the bundles must contain entire second stage nodes
so the resulting bundles represent a two-stage problem.