Location-> dismod_at release_notes whats_new_2017

Search-> contents reference index search external

Prev

Next

whats_new_2017

Heading-> mm-dd ---..12-30 ---..12-22 ---..12-21 ---..12-20 ---..12-16 ---..12-13 ---..12-10 ---..11-12 ---..11-08 ---..10-25 ---..10-24 ---..10-22 ---..10-21 ---..10-20 ---..10-18 ---..10-17 ---..10-14 ---..10-13 ---..10-09 ---..10-06 ---..10-05 ---..10-04 ---..10-02 ---..10-01 ---..09-30 ---..09-29 ---..09-28 ---..09-27 ---..09-26 ---..09-25 ---..09-23 ---..09-21 ---..09-20 ---..09-19 ---..09-18 ---..09-17 ---..09-16 ---..09-02 ---..09-01 ---..08-30 ---..08-07 ---..05-23 ---..05-05 ---..05-02 ---..05-01 ---..04-29 ---..04-27 ---..04-26 ---..04-25 ---..04-24 ---..04-23 ---..04-22 ---..04-21 ---..04-20 ---..04-19 ---..04-18 ---..04-17 ---..04-16 ---..04-15 ---..04-09 ---..04-07 ---..04-05 ---..04-04 ---..04-03 ---..04-01 ---..03-30 ---..03-29 ---..03-17 ---..03-06 ---..03-04 ---..03-03 ---..01-27 ---..01-26 ---..01-24 ---..01-14 ---..01-15

@(@\newcommand{\B}[1]{ {\bf #1} } \newcommand{\R}[1]{ {\rm #1} } \newcommand{\W}[1]{ \; #1 \; }@)@ This is dismod_at-20221105 documentation: Here is a link to its current documentation . Changes and Additions to Dismod_at During 2017
mm-dd

12-30
Advance to using cppad_mixed-20171230. This uses the CppAD subgraph method to compute the Hessian with respect to the random effects. It is expected to be faster on large problems, but more testing is required to verify this.

12-22
Some users did not realize that bin/install_all.sh was just an example install to help system administrators. This script has been included in the documentation and moved to bin/example_install.sh to make this clear.

12-21
Advance to using cppad-20171215 which is required by cppad_mixed-20171218.

12-20

1. Advance to using cppad_mixed-20171218. This uses the CppAD subgraph method to compute the sparsity of the Hessian with respect to the random effects. It is expected to be faster on large problems, but more testing is required to verify this.
2. Sometimes, new versions of the python files were not installed because the bin/setup.py file was not executable. This has been fixed.

12-16

1. There was a mistake in the linking of the sqlite3 and gsl libraries that caused a problem during testing on some systems. This has been fixed.
2. The value libdir is now specified in bin/run_cmake.sh (it used to be specified by bin/libdir.sh).

12-13
There was a bug in the predict sample command when build_type was debug. To be specific, the check to see if the database was consistent had an error in it. This has been fixed.

12-10

1. Move documentation from IHME to github; see
https://bradbell.github.io/dismod_at/doc/index.html .
2. The git program was added to the system requirements.
3. The download instructions were changed to use git for distributing the program.

11-12

1. Fix the definition of the adjusted standard deviation for the measurement values (the definition for log an linear cases is the same).
2. Fix the definition of the meas_value covariate multipliers beta_j and the meas_noise covariate multipliers gamma_j (a, t) . They used to be @(@ \beta_{i,j} @)@ and @(@ \gamma_{i,j} @)@ but they do not depend on the data index @(@ i @)@.

11-08
Change the title of the data_like section and fix minor typos in the notation for the age upper limit b_i and time upper limit t_i .

10-25

1. Advance to cppad_mixed-20171025 and install it using the rev_hes_sparsity option.
2. Advance to cppad-20171023 to avoid a compiler warning.

10-24
Some minor changes were made to the bilinear interpolation section. To be more specific, some typos were fixed in function grid and the outside both limits discussions. In addition, the plotting discussion was added.

10-22

1. Add the option to specify parent_node_name instead of parent_node_id. The create_database routine was changed to put the parent_node_name in the option table, instead of mapping it to the parent_node_id. (Using parent_node_id avoids a lookup in the node table, but the time for this lookup is not expected to be significant.)
2. Fix create_connection in case where new is true and previous version of the file did not exist. (Not sure why this did not create more failures.)

10-21

1. The depend_command was added and the depend column was added to the variable.csv file.
2. The Input and Output by Command paragraph in the data_flow page was corrected. To be specific fit_data_subset_table , not data_subset_table , is output by the fit_command .

10-20
Add the checkpoint_newton_step option to the dismod_at configuration script. The default value in run_cmake.sh for this setting is yes (it used to be set at no).

10-18

1. Improve documentation for following sections: avg_integrand , model_variables , get_started_db.py .
2. Add cross references from values in the rate_table and integrand_table to the corresponding average integrand notation.
3. Simplify the python code in get_started_db.py .
4. A warning about re-running the set_command command was added to the set command and the modify_command .

10-17

1. The db2csv_command.py example / test was added and the corresponding wish_list item was removed.
2. The log messages were missing for the db2csv_command . This has been fixed.
3. The empty text was sometimes placed in the table_name column of the log table. This was a bug and has been fixed by using null in these cases; see text can be null, not empty .
4. The create_connection routine now exist with a fatal error message when the specified file_name does not exist.
5. The modify_command was added.

10-14

1. Specifications were added to forbid child data from having laplace or log_laplace densities.
2. In addition, the discussion of random_effects vector @(@ u @)@ in the data likelihood model was improved.

10-13

1. In the variable.csv file replace the sam_value column by the sam_avg and sam_std columns.
2. The simulate_command has been extended to handle students and log_students densities in the density_id .

10-09
Advance to cppad_mixed-20171009.

10-06

1. Add the truth column to the variable.csv file create by the db2csv_command .
2. Add the sample table as a possible source for the values used in the set command .
3. Add the fit_var and truth_var tables as a possible source for the predict command. Note that the predict command now requires the source argument.
4. The fit_var method for the sample command has been removed. It was no longer useful given the ability to specify fit_var in the predict_command .
5. In the fit table, change variable_value to fit_var_value .
6. There was a problem with the install process where by the include directories for sqlite3 , gsl , or ipopt , might not be set correctly. This has been fixed.

10-05

1. The restrictions on the child value priors and child difference priors have been changed. To be specific, the density need not be Gaussian, it can be anything but Laplace and Log-Laplace. The user needs to be careful to ensure that the Hessian of the likelihood with respect to the random effects is positive definite; see random effects Hessian .
2. Add specifications for comment tables .
3. Change the create_database so that when certain entries are not present, the value null is used for the corresponding entry in the database. In addition, remove the corresponding setting to None in the user_example examples.
4. Change random_bound to bound_random . Change zero_sum_child_rate to zero_sum_child_rate . Change fixed_bound_frac to bound_frac_fixed .

10-04
Add the Student's-t and Log-Student's-t density functions; see user_students.py. This required changing density_table in a way that is not backward compatible. To be specific, the students and log_students rows were added to the table.

10-02

1. Change the create_database data_table so that meas_std and eta need not appear when they are null. In addition, remove the null eta values from the user_example examples.
2. Reduce the chance of random failure in the user_lasso_covariate.py example.

10-01
Simplify the weighted residual function R and the log density function D . To be specific, the density_id and eta were combined into @(@ d @)@ in the

09-30
Advance to cppad_mixed-20170930. This fixes a bug where the optimization of the fixed effects was terminating (instead of backing up) when the optimization of the random effects failed.

09-29

1. The change on 09-20 introduced a bug because it ignored the warn_on_stderr option. This has been fixed.
2. The user_re_scale.py example was added.
3. Add the option version of the set command. Changes to the option table that are done this way get recorded in the log_table .

09-28
Add the scale_var table which enables the user to have more control over scaling the objective. See scale_var_value and set command .

09-27

1. This change was merged into the master branch on 09-29. Remove the start command. It has been replace by the set_command syntax
   dismod_at set start_var source

2. Fix a bug in db2csv_command that occurred when mulstd_value_prior_id was not null.
3. Fix a bug in db2csv_command when warn_on_stderr was present.
4. Fix a bug in db2csv_command when data_extra_columns or avgint_extra_columns was present and null.
5. A db2csv wish list item was added (fulfilled on 10-17 ).

09-26
These changes were merged into the master branch on 09-29.

1. Add the sql_command python function.
2. Change init_command so that it also creates a version of the start_var_table using the prior means.
3. Remove the truth command. It has been replace by the set_command syntax
   dismod_at set truth_var fit_var


09-25
Advance to cppad_mixed-20170925 for improved error handling and messaging.

09-23
Advance to cppad_mixed-20170923 for improved error handling and messaging.

09-21

1. Advance to cppad_mixed-20160919 We are working on improving error messaging when nan occurs during optimization.
2. Add susceptible and withC to the documentation for the types of average integrands I_i(a,t) . These integrands were already available, see integrand_name .

09-20
All warnings are now printed to standard error as well as being logged in the database. Errors are also printed, but this is not a change.

09-19

1. Some of the system requirements were not being installed in the proper place on the MAC. The install has been modified to use pkg-config to find these non-standard locations.
2. The log_fatal_error option was removed from run_cmake.sh (by putting an assert before the exit in the fatal error routine).

09-18

1. Advance to cppad_mixed-20160918; see cppad_mixed . This provides more error messaging when nan's occur in the random effects objective.
2. The documentation for bounds was improved.

09-17

1. There was a bug in setting the lower and upper limits of random effects equal that sometimes caused a nan result during initialize. This has been fixed.
2. Make bilinear and ode_grid separate sections and improve their presentations.

09-16
Advance to cppad_mixed-20160916; see cppad_mixed . This makes the optimization with respect to the random effects back up when the object results in a nan.

09-02

1. Instructions were added for building both the debug and release debug and release version of dismod_at on the same system.
2. The cppad_mixed install script was changed so that it does not optimize AD functions when build_type is debug. This can be used to verify that optimizing to AD functions only affects the speed and memory used by dismod_at and not its results.

09-01

1. Add the n_children argument to the speed example / test.
2. Add a definition of the children to the option table.

08-30
Advance to cppad-20170830. This fixes a problem with the cmake command; to be specific,
   Policy CMP0046 is not set: Error on non-existent dependency

08-07

1. Advance to cppad_mixed-20170801 to fix an install problem where ran_obj_tst was not defined.
2. The postscripts fixed and random were missing from the accept_after_max_steps documentation.
3. It would be nice to have a sample_data_subset table that is similar to the fit_data_subset_table , but corresponds to sample command with method equal to data_sim.

05-23
Advance to cppad-20170522.

05-05
Improved the detection of which data_table rows do not require the random effects to make correspond avg_integrand model prediction. This may make the computation more efficient, but otherwise the user should see no difference.

05-02
The noise covariate multipliers effect used to scale the measurement standard deviation. Now it is directly added to the measurement standard deviation; see adjusted standard deviation . This makes the meas_noise covariate multipliers more difficult to use because they have units. On the other hand, it is more flexible for use in meta-regression when the standard deviations for some measurements is small and the difference between measurement sets is large.

05-01

1. Remove recommendation for using prior_zero and prior_one as prior names . These have been replace by more flexible, and easier to use const_value , and use of null as smoothing; e.g., null as a parent_smooth_id in the rate table .
2. Provide more information by splitting node into child and node in data.csv
3. The get_started section has been moved to the same level as user_example and the Commands heading has been added to the user section (it has since been removed).
4. Improve smooth_grid table documentation for cases and meaning when the following value are null: const_value , value_prior_id , dage_prior_id , dtime_prior_id .

04-29

1. The avgint_subset table has been removed. The avgint table is only used by the predict_command . As such, it can be changed, and the predict command can be re-run, without having to first run any other commands.
2. There was a bug and the case fit fixed would not run when zero_sum_child_rate was not null (specified one or more rates). This has been fixed.
3. For each rate_name in zero_sum_child_rate the corresponding child_nslist_id must be null. An error messages is reported if this is not the case.

04-27
Change the zero_sum_child_rate option so it can select specific rates to have zero sum. This was necessary so that child_nslist_id can be used to use different priors for each child and some rates, while other rates have the same prior for each child and zero sum.

04-26

1. In the user_example examples, move the setup of avgint_table just before the setup of data_table because they are so similar.
2. The avgint_id was added to the predict.csv file.
3. Add data_extra_columns and remove data_name column from the data table.

04-25

1. Advance to cppad_mixed-20170424.
2. The convergence criteria, is now relative to the derivative values at the mean of the prior for the fixed effects. This also holds for the values printed during the optimization of the fixed effects, see print_level . These values used to be relative to the starting variables which would change when one continued an optimization.
3. Add the user_continue_fit.py example.
4. Add the warn_on_stderr option.
5. Add the avgint_extra_columns option.

04-24
Fix bug in dev::eigen_ode2.cpp where the std::abs function was being called and std::fabs should have been called. (This caused some tests to fail on the Mac but not on Fedora.)

04-23
Advance to cppad_mixed-20170423, this improves the detection of convergence of the fixed effects optimization problem, see fit_command .

04-22

1. The init_command now deletes any existing output tables .
2. The version number in the dismod_at python package was stuck at 20160516; i.e., not being updated. This has been fixed.

04-21

1. Change the name of the output section to the data_flow section.
2. Add the minimum_meas_cv option. This is useful because the IHME database has bad measurement standard deviations that are two or more orders of magnitude small than the corresponding measurement values.
3. Change the simulate command so only the log_gaussian and log_laplace negative data is set to zero. (It used to also do this in the gaussian and laplace cases.)

04-20

1. Add a comment about changing values , in the input tables, to the init command.
2. Add more links that show how the different model_variables appear in the avg_integrand .
3. Correct the discussion of model variables to state that it is now possible to have a different smoothing for each child rate effect .

04-19
The offset in the log transformation for the data is now specified in the data table (for each data point) instead of in the integrand table (for all data points corresponding to an integrand); see eta .

04-18
Change the definition of adjusted standard deviation for data in the Gaussian and Laplace case (it is now different from the Log-Gaussian and Log-Laplace case). (This has since been changed back.)

04-17
Add random to possible choices for variables during the fit.

04-16

1. The change on 04-15 introduced a bug whereby non-zero values for bound_random were not having any effects. This has been fixed.
2. The variables argument was added to the fit command. It is better to use variables equal to fixed, a apposed to changing bound_random in the option table.
3. The wish_list was updated.

04-15
It is now possible to constrain each random effect to have a specific value; see constant child value priors .

04-09

1. The const_value option was added to the smooth_grid table. This makes it much easier to constrain a variable to have a specific value at a specific age and time.
2. Instructions on how to create the database example.db for a specific example, were added to the user example section.

04-07

1. More discussion was added for how the parent node and the max_difference affect the data_subset_table and avgint_subset_table.
2. The get_started example was changed to use a single exponential model. In addition, the get_started_db.py example is now a good starting point for editing to add more complexity to the model.

04-05

1. The definition of the average integrand has been changed to include the measurement value covariate effect .
2. The bin/install_suitesparse.sh script has been changed to work on the MAC (where the -lrt library is not available).
3. The optimization tolerance in user_group_mulcov.py was made smaller to avoid test failures on some machines.

04-04
The notation average weight (for @(@ \bar{w}_i @)@) was corrected to be weight integral .

04-03
Advance to cppad-20170402 and cppad_mixed-20170403. The Cppad change fixes a bug in the optimization of conditional expressions. This in turn enables Cppad Mixed to make more use of AD function optimization (and still pass all the tests in dismod_at).

04-01

1. Remove the need to set PKG_CONFIG_PATH during the install_unix procedure (dismod_at_prefix in bin/run_cmake.sh is used to determine this setting automatically).
2. Improve the discussion on setting the python path .

03-30
Add susceptible and withC to the list of possible integrand names; see integrand_name .

03-29
Advance to cppad-20170329 and cppad_mixed-20170329.

03-17
Advance to cppad_mixed-20170317. This includes a change to the cppad_mixed, but no change from the dismod_at user's perspective.

03-06
Advance to cppad-20170306 . This fixes another bug in Cppad.

03-04
Advance to cppad-20170304 . This fixes a bug in Cppad. Advance to cppad_mixed-20170304 . This improves the scaling and solution check for optimization of the fixed effects.

03-03
Simplify the install procedure. Now it is only necessary to adjust the settings in run_cmake.sh . These settings are automatically used by all the bin/install_package.sh scripts.

01-27
The values in the covariate columns of the predict.csv file were not correct (did not correspond to the avgint_table . This has been fixed.

01-26
Advance to cppad_mixed-20171126. This version includes the scaling mentioned under 01-24, but prints the original (unscaled) objective in the optimizer trace.

01-24
Advance to cppad_mixed-20171124. This version re-scales the problem using the derivatives values at the starting point. This is in addition to the scaling done using eta.

01-14
The adaptive and trace-adaptive derivative_test options generated error messages. This has been fixed and they now work.

01-15
Advance to cppad-20161113 and cppad_mixed-20170114.

---

Input File: omh/whats_new/2017.omh