• Unified framework for both uncontrained and constrained optimization
• Efficient and empowered by automatic differentiation engine of Pytorch
• Use-friendly and modifiable whenever you want
• Visualizable for both surface (or curve) and contour plot

$ git clone ...
$ cd MagiOPT

• Unconstrained

    import MagiOPT as optim
    
    def func(x):
        ...
        
    optimizer = optim.SD(func) # Steepest Descent
    x = optimizer.step(x0)     # On-the-fly
    optimizer.plot()           # Visualize

• Constrained

    import MagiOPT as optim
    
    def object(x):
        ...
    def constr1(x):
        ...
    def constr2(x):
        ...
    ...
    
    optimizer = optim.Penalty(object, 
                              sigma, 
                              (constr1, '<='), 
                              (constr2, '>='), 
                              plot=True))       # Penalty methoed
    optimizer.BFGS()                            # Inner optimizer
    x = optimizer.step(x0)                      # On-the-fly

Use a simple line of code for unconstrained optimizer

optimizer.plot()

we can visualize the 2D curve with iterated sequence. Such that

+------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+ | .. image:: https://github.com/MagiFeeney/MagiOPT/blob/859f80525d66a1d6799024721b8fb5a508a9ae6f/MagiOPT/examples/sd/Figure_sd_2.png | .. image:: https://github.com/MagiFeeney/MagiOPT/blob/859f80525d66a1d6799024721b8fb5a508a9ae6f/MagiOPT/examples/sd/Figure_sd_3.png | +------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+

| | |

Or use

optimizer = optim.Penalty(..., plot=True)

we can visualize the function and sequence of each inner iteration with the 3D surface with iterated sequence, and its contour with iterated sequence.

| ------ | ------ | | | |

| | |

| | |

| | |

| | |

| | |

• Majority of algorithms are sensitive to initial point; choosing properly will save a lot of your effort
• Due to ill-conditioned situation, constrained optimizer may need you to trial-and-error
• The behavior of Barzilai-Borwein method is not stable for non-qudratic problem, however, you can still infer path through an intermediate visualization
• You can extract the sequence easily by

  optimization.sequence

• Pytorch 3.7 or above