Prev Next whats_new_2016

@(@\newcommand{\B}[1]{ {\bf #1} } \newcommand{\R}[1]{ {\rm #1} } \newcommand{\W}[1]{ \; #1 \; }@)@ This is dismod_at-20221105 documentation: Here is a link to its current documentation .
Changes and Additions to Dismod_at During 2016

mm-dd

12-03
  1. The meas_std filed was added to the data_sim table.
  2. The option arguments were added to each command message in the log table.
  3. The db2csv_command was modified so that meas_value and meas_std always correspond to the data_table . The columns sim_value and sim_std correspond to the data_sim_table when the previous fit used simulated data. In addition, some bugs in this command, related to the simulate_index , were fixed.
  4. There was some confusion in the documentation of the columns in the predict.csv file. To be specific, it was not clear that these columns correspond to the avgint_table instead of the data_table . This has been fixed.
  5. The message_type warning was missing in the log file documentation. It has been added.
  6. Change the simulate_command so that it adjusts meas_std data_sim table field to be relative to the simulated measurements instead of using the data table meas_std .


10-31
  1. Include documentation for the covariate columns in the data.csv file.
  2. Improve the documentation for the sample_index column in the predict table. (It refers to variable and not data samples.)
  3. Add the predict.csv output file to the db2csv command.


10-20
  1. API Change: The fit_simulate_index option was removed from the option_table and the argument simulate_index was added to the simulate command.
  2. Improve the discussion of the output tables .
  3. Fix some compiler warnings.


10-19
  1. API Change: The argument number_sample was added to the sample command.
  2. API Change: The number_simulate option was removed from the option_table and the argument number_simulate was added to the simulate command.
  3. The log.csv file was added too the db2csv command output.


10-18
  1. In the user_example examples, change name_dict to name_table for name equal to var, predict, and sample. This makes it clearer that they are copies of the corresponding tables as a python list of dictionaries; see get_table_dict .
  2. If bound_random was zero, the sample_command would terminate with a cppad_mixed error message. This has been fixed. In addition, the random bound documentation now includes mention that is not accounted for in the asymptotic .
  3. Advance to cppad_mixed-20161018. This fixed a bug in cppad_mixed logging error messages versus generating asserts.


10-17
  1. Improve the invalid prevalence error message.
  2. Fix some more warnings on a old compiler (g++4.4.7).
  3. Add choice for debug or release build in the IHME cluster program install_dismod_at.sh.
  4. In cmake, search the default library directory last. This enables choosing a non-default version of a library as well as finding the default libraries when needed.
  5. Fix assertion when log_fatal_error is false.
  6. Add the system_specific_library_list argument to the cmake install script.


10-16
  1. The zsum_child_rate.py example / test was added.
  2. There was a bug (that has been fixed) in create_database . This bug would sometimes mark the wrong time index as having a null dtime_prior_id in the smooth_grid table.
  3. Add the log_fatal_error flag to the dismod_at configuration.
  4. Remove the dev::pack_info.cpp variable_name routine. (The var_table and variable.csv file provide this information.)
  5. Report invalid prevalence values in the log_table (in the case were log_fatal_error is true).


10-15
Add the zero_sum_child_rate option which constrains the sum of the random effects to be zero.

10-14
  1. Fix a warning (that appeared on some systems) about missing braces in devel/dismod_at.cpp.
  2. Add option so that user can specify the C++ compiler during the install of ipopt and cppad_mixed
  3. API Change: the data_name field was added to the data table.


10-13
  1. Improve bin/install_suitesparse.sh so that it automatically includes the metis package. Note that the proper value for metis_version , in the suitesparse install, is printed at the end of the ipopt install.
  2. Improve the way that cmake searches for libraries and reports failure to find a particular library. The search was failing on a particular Mac system.
  3. Advance cppad-20161012. This fixes a problem in the detection of c++ 2011 feature availability.


10-12
  1. Add the option to set the bound_random to zero.
  2. Change name_dict to name_table in the python examples. For example, data_dict was changed to data_table .
  3. There was a problem with the cppad install script bin/install_cppad.sh that caused a compiler error in forward2sweep.hpp when CppAD was not using c++11 features. This has been fixed.


10-11
The import_cascade.py program was removed (it has not been kept up to date).

09-30
  1. A new assert, that checks for infinity or nan results during the integration, was added. This should be converted to a more useful error message; (this was done on 10-16 ). Furthermore, in some cases it can be avoided; see large excess mortality in the wish list.
  2. Use one value for libdir for all the install scripts; determined by bin/libdir.sh.
  3. Change run_cmake.sh to use link from build and install prefix to corresponding debug or release directories (so same as install scripts).
  4. Advance to Ipopt-3.12.6


09-29
  1. In the case where dismod_at is in the prefix, build debug and release versions in separate directories; see special requirements .
  2. Advance to eigen-3.2.9 and cppad-20160929.
  3. There was a bug in the sampling of the random effects during the sample asymptotic command (found by a gcc-6.2.1 -Wmisleading-indentation warning). This has been fixed.
  4. Add a missing import sys to db2csv_command.py.


09-21
Add the option.csv table to the output generated by the command
   dismodat.py database db2csv
08-28
Change 'If the fit_command has not been run' to 'If the fit_command has been run' in db2csv_command . Make the same change for the reference to running the sample_command.

07-29
  1. Add the optimizer accept_after_max_steps option to both the fixed and random effects optimization.
  2. Correct a problem with detecting the actual random seed in the user_speed.py speed test.


07-28
  1. Advance to cppad_mixed-20150728. This should increase the speed of the fit_command . It should also make it more robust; i.e., work in more cases.
  2. The user_speed.py example now uses the fit_command , instead of the sample_command to fit the simulated data.
  3. Add comments about debug and release builds under build_type in run_cmake.sh.
  4. Change DEBUG and RELEASE to debug and release in run_cmake.sh and the install for eigen , suitesparse , ipopt , cppad_mixed .


07-15
In bin/install_cppad_mixed.sh change cmake_build_type to build_type; see cppad_mixed . In addition, add this parameter to the install for eigen , suitesparse , and ipopt . CppAD does not need this choice as it adapter to the choice of the systems that use it. Note that the install for dismod_at must have the same build_type as cppad_mixed . (This is no longer an issue.)

07-14
Advance to cppad_mixed-20150711. This should increase the speed of the asymptotic version of the sample_command. It also includes a change to optimizing the random effects that should eventually improve the speed of fit_command . (At this point, the change does not always result in an improvement.)

07-11
Advance to cppad_mixed-20150711. This should give better recovery when the fixed effects enter a region where the Hessian w.r.t the random effects it is not positive definite. To be specific, the optimizer should try backing up the fixed effects and if that fails, it should exit the optimization and report the value of the fixed effects.

06-13
Advance to cppad_mixed-20150613. This fixes a bug in the asymptotic version of the sample command.

06-12
Advance to cppad_mixed-20150612. This removes the choice between ipopt and box_newton for optimizing the random effects; see 06-05 below.

06-06
Advance to cppad_mixed-20150606.

06-05
Advance to cppad_mixed-20150605. This involved hooking up new option to choose between ipopt and box_newton for optimizing the random effects; see random_box_newton. The user_speed.py speed test time (on one machine) changed from elapsed seconds = 68.09 to elapsed seconds = 42.31 when random_box_newton was changed from false (the default) to true. The random seed for both cases above was random_seed = 1465142803. The results for the two cases were the same.

05-16
  1. The database2csv program has been moved to the db2csv_command .
  2. The python_three_command installation setting has been changed to the python3_executable setting.
  3. The dismod_at python module is now installed during the make install command. The dismodat.py program is also installed by this command; e.g. see db2csv_command .


05-15
  1. Advance to cppad_mixed-20150515 This yields better sample_command results when the implicit information matrix is not positive definite (for diagnostic purposes).
  2. Add the adaptive and trace-adaptive values for the derivative_test (derivative_test_fixed only).


05-11
  1. Advance to cppad_mixed-20150511. This yields sampling when covariance matrix is not positive definite (for diagnostic purposes).
  2. Add sampling random effects to the wish list items.
  3. Add a description of the age and time variation for each of the model variables.


05-10
Fix db2csv_command so that it works with smoothing standard deviation multipliers . This required adding the s_id column to the variable.csv file.

05-09
  1. Change the scaling of fixed effect to key off eta not null in value priors; see scaling . This enables more user choice over which fixed effects get scaled.
  2. The error detection for non-positive Hessian (added yesterday) was working properly when variables were bound constrained.
  3. The sample asymptotic command was using the scaled information matrix. This has been fixed.
  4. Advance to cppad_mixed-20160511.


05-08
  1. Advance to cppad_mixed-20160508. This enables one to use asymptotic statistics when the information matrix is singular, by setting the lower and upper limits equal for variables that have non-positive Hessian.
  2. This new version of cppad_mixed also has new random effects simulator that uses less memory and less time.
  3. During the sample_command , detect when the Hessian with respect to a fixed effect is not positive and report the corresponding var_id .


05-07
  1. Fix a bug that occurred when the minimum (maximum) value in the age_table and time_table was not the first (last) entry.
  2. Add a db2csv_command item to the wish list.
  3. Start work on test/diabetes.py (under construction).


05-06
Advance to cppad_mixed-20160506 to fix a bug in sample asymptotic when there are no random effects.

05-05
  1. Change the max_num_iter option so that -1 returns the input model_variables , and 0, includes the move of the initial variable values (to be inside the constraints).
  2. Change the option_table so that all entries have default values and do not need to appear in the table.
  3. Add the bound_frac_fixed option.


05-04
  1. Print the ipopt library directories during the cmake command .
  2. In the db2csv_command program, change fit_res, fit_dage and fit_dtime to res_value res_dage and res_dtime respectively.
  3. Put cppad_mixed: at the beginning of warnings and error messages that come from the cppad_mixed package.
  4. Add missing dev::log_message::message_type cases to the log_message documentation.
  5. Change user_group_mulcov.py to use null smoothing pointer in the middle (not last) covariate multiplier (better test).
  6. Change test/user/relrisk.py tolerance so that the cppad_mixed solution check passes.
  7. Advance to cppad_mixed-20160504.
  8. Change the fixed effects optimization so that it uses the Ipopt option nlp_scaling_method = none . This fixed the remain solution check warnings generated by cppad_mixed.


05-03
  1. Advance to cppad_mixed-20160503.
  2. Fix warning in devel/utility/pack_info.cpp.


05-01
  1. Allow for a covariate multiplier group_smooth_id to be null. This enables on the include, or drop out, a multiplier by only changing one entry the mulcov_table .
  2. Fix problem with documentation for sigma that occurred in changes on 04-22 .
  3. Fix discussion of lambda_j in the value prior for fixed effects. To be specific, all fixed effects @(@ \theta_j @)@, not just the parent rates, have a value prior multiplier @(@ \lambda_j @)@.


04-28
  1. Add the metropolis MCMC utility for testing results.
  2. Complete implementation and testing of the sample command with method equal to asymptotic; see the example and test in user_sample_asy.py .


04-27
  1. Advance to cppad_mixed-20160427.
  2. Add discussion about bounds and Lagrange multipliers .
  3. Remove dismod_at::manage_gsl_rng and instead use CppAD::mixed::manage_gsl_rng. This fixed that use of random_seed so that it applies to all dismod_at simulated values.
  4. Add a choice of method to the sample command. This makes it easier to predict with the model_variables equal to the results of a fit. It also provides asymptotic posterior sampling for the model_variables.
  5. There were some place in the code where dismod_at was assuming that the minimum and maximum age and time were the first and last entries in the age_table and time_table respectively. This has been fixed.


04-24
  1. The db2csv_command utility has been documented and is now part of the user API.
  2. The name of the table section has been changed to database .


04-23
  1. Change the data weighted residual to be on the measurement values, not the adjusted measurement values. This simplifies the weighted residuals and avoids the condition where the mean_value covariate multipliers affect the adjusted standard deviation .
  2. Add the Lagrange multipliers lagrange_value , lagrange_dage and lagrange_dtime to the fit_var table.
  3. Advance to cppad_mixed-20160423.


04-22
  1. Scale (during optimization of fixed effects) all the model_variables that have a log-Gaussian or log-Laplace distribution for their value prior. To be specific, the optimization variables are the offset log transform of the corresponding model variables.
  2. When there was no data, and prior for the model_variables was log-Gaussian or log-Laplace, the prior mean was not the optimal estimate. This was because the estimate for the model variance affected the log-transformed standard deviation sigma . This was a feature (it agreed with the documentation) but it was not desirable. This have been changed so that sigma is defined in terms of the prior mean and not in terms of the model value. The corresponding log-Gaussian wish list item has been removed.


04-21
Change the sample command so that it has a method argument. Using method = simulate does the same as the sample command before this change. The new feature is method = fit_var . The asymptotic method is not yet implemented.

04-20
  1. Change number_sample in option table to number_simulate .
  2. The method argument was added to the sample command.


04-19
  1. In the data_sim table, change sample_index to simulate_index .
  2. In the option table, change fit_sample_index to fit_simulate_index .


04-18
Advance to new version of CppAD and cppad_mixed this fixes the compiler warnings when building the release version.

04-17
Fix some errors and warnings when build_type was RELEASE. Note that release version is about twice as fast as the debug version (when using the debug version of cppad_mixed .

04-11
Initialize the optimization when max_num_iter_fixed is zero, (this does more model checking).

04-09
Advance to a newer version of cppad_mixed. This has a change so that when max_num_iter is zero, for the fixed effects optimization problem, the resulting fixed effects estimate is the same as in the start_var_table .

04-07
Advance to a newer version of cppad_mixed (rerun bin/install_cppad_mixed.sh). This has a change to the routine that checks derivatives of the fixed effects optimization problem.

03-29
Advance to a newer version of cppad_mixed (rerun bin/install_cppad_mixed.sh). This has an important bug fix (see cppad_mixed whats new).

02-26
Advance to a newer version of cppad_mixed (rerun bin/install_cppad_mixed.sh). This has an important bug fix (see cppad_mixed whats new).

02-06
Advance to a newer version of cppad_mixed (rerun bin/install_cppad_mixed.sh).

01-26
  1. Advance to a newer version of cppad_mixed (rerun bin/install_cppad_mixed.sh).
  2. The SuiteSparse package is now required. This is because one now has the option to use it for Cholesky factorization; see the cholmod_cholesky setting for installing cppad_mixed .


01-16
Remove some wish_list items that have been completed and move some others to the cppad_mixed package.

01-15
Advance to newer version of cppad_mixed.

01-10
Advance to newer version of Eigen and cppad_mixed. You should rerun bin/install_eigen.sh and then bin/install_cppad_mixed.sh.

01-06
The avgint_table created by import_cascade.py had the wrong number of age values. This has been fixed.

01-05
Advance to a newer version of cppad_mixed (rerun bin/install_cppad_mixed.sh).
Input File: omh/whats_new/2016.omh