htmd.adaptive.adaptivebandit module#

class htmd.adaptive.adaptivebandit.AdaptiveBandit#

Bases: AdaptiveBase

Adaptive sampling using a bandit algorithm for state selection.

Uses an Upper Confidence Bound (UCB) strategy over a Markov state model to balance exploration and exploitation when choosing respawning conformations.

Parameters:

  • app (SimQueue object, default None) – A SimQueue class object used to retrieve and submit simulations

  • project (str, default 'adaptive') – The name of the project

  • nmin (int, default 0) – Minimum number of running simulations

  • nmax (int, default 1) – Maximum number of running simulations

  • nepochs (int, default 1000) – Stop adaptive once we have reached this number of epochs

  • nframes (int, default 0) – Stop adaptive once we have simulated this number of aggregate simulation frames.

  • inputpath (str, default 'input') – The directory used to store input folders

  • generatorspath (str, default 'generators') – The directory containing the generators

  • dryrun (boolean, default False) – A dry run means that the adaptive will retrieve and generate a new epoch but not submit the simulations

  • updateperiod (float, default 0) – When set to a value other than 0, the adaptive will run synchronously every updateperiod seconds

  • coorname (str, default 'input.coor') – Name of the file containing the starting coordinates for the new simulations

  • boxname (str, default 'input.xsc') – Name of the file containing the starting box dimensions for the new simulations. Set to ‘none’ to disable box writing.

  • lock (bool, default False) – Lock the folder while adaptive is ongoing

  • mps (int, default 0) – If mps > 0, it will run simulations using the Multi-Process Service (MPS) with the number of processes specified. If set to 0, mps is disabled

  • datapath (str, default 'data') – The directory in which the completed simulations are stored

  • filter (bool, default True) – Enable or disable filtering of trajectories.

  • filtersel (str, default 'not water') – Filtering atom selection

  • filteredpath (str, default 'filtered') – The directory in which the filtered simulations will be stored

  • projection (Projection object, default None) – A Projection class object or a list of objects which will be used to project the simulation data before constructing a Markov model

  • goalfunction (function, default None) – This function will be used to convert the goal-projected simulation data to a ranking whichcan be used for the directed component of FAST.

  • reward_method (str, default 'mean') – The reward method

  • statetype ((``’cluster’, ``'micro', 'macro'), str, default 'micro') – State type (cluster, micro, macro) to use for reward calculations.

  • skip (int, default 1) – Allows skipping of simulation frames to reduce data. i.e. skip=3 will only keep every third frame

  • lag (int, default 1) – The lagtime used to create the Markov model. Units are in frames.

  • exploration (float, default 0.01) – Exploration is the coefficient used in UCB algorithm to weight the exploration value

  • temperature (int, default 300) – Temperature used to compute the free energy

  • ticalag (int, default 20) – Lagtime to use for TICA in frames. When using skip remember to change this accordinly.

  • ticadim (int, default 3) – Number of TICA dimensions to use. When set to 0 it disables TICA

  • clustmethod (ClusterMixin class, default <class 'sklearn.cluster._kmeans.MiniBatchKMeans'>) – Clustering algorithm used to cluster the contacts or distances

  • macronum (int, default 8) – The number of macrostates to produce

  • save (bool, default False) – Save the model generated

  • save_qval (bool, default False) – Save the Q(a) and N values for every epoch

  • actionspace ((``’metric’, ``'goal', 'tica', 'ticapcca'), str, default 'tica') – The projected space which will be clustered and used to define our action space

  • recluster (bool, default False) – If to recluster the projected space used in MSM estimation with a different clustering.

  • reclusterMethod (, default <class 'htmd.clustering.regular.RegCluster'>) – Clustering method for reclustering.

  • goal_init (float, default 0.3) – The proportional ratio of goal initialization compared to max frames set by nframes

conformationStationaryDistribution(model)#
Return type:

list

count_pucb(q_value, exploration, predictor, step, n_value)#

Compute the predictive UCB value for a state.

Parameters:

  • q_value (float) – Current estimated reward for the state.

  • exploration (float) – Exploration coefficient weighting the confidence bound.

  • predictor (float) – Predictor value scaling the exploration bonus.

  • step (int) – Total number of steps taken so far.

  • n_value (int) – Number of times the state has been visited.

Returns:

pucb – Predictive UCB score for the state.

Return type:

float

count_ucb(q_value, exploration, step, n_value)#

Compute the UCB value for a state.

Parameters:

  • q_value (float) – Current estimated reward for the state.

  • exploration (float) – Exploration coefficient weighting the confidence bound.

  • step (int) – Total number of steps taken so far.

  • n_value (int) – Number of times the state has been visited.

Returns:

ucb – UCB score for the state.

Return type:

float

getRewards(trajidx, data_q, confstatdist, numstates, rewardmethod)#
Return type:

list