Prev Next whats_new_2020

@(@\newcommand{\B}[1]{ {\bf #1} } \newcommand{\R}[1]{ {\rm #1} } \newcommand{\W}[1]{ \; #1 \; }@)@ This is dismod_at-20221105 documentation: Here is a link to its current documentation .
Changes and Additions to Dismod_at During 2020

mm-dd

12-30
The prior_sim_table , hes_fixed_table , and hes_random_table output tables were not being dropped (if they existed) during the init_command . This has been fixed.

12-29
Fix a problem when creating both debug and release builds. (This was not a problem before 12-22 when build/external moved to external).

12-26
  1. Change bin/db_simplify.py from a Chron's Disease to a Diabetes case.
  2. Add a log density item and a speed item to the wish list. Remove the checkpointing item ,and change the conditional item heading to rate_case .


12-22
Advance to cppad_mixed-20201222. This improves the error messaging during the install of cppad_mixed.sh . In addition, this new version avoids ipopt creating of the file debug.out (when build_type is debug).

12-17
  1. The example user_sum_residual.py discusses why the sum of the weighted residuals is not expected to be zero.
  2. Under the heading PKG_CONFIG_PATH the command option --version was corrected to --modversion.
  3. The scripts bin/csv_summary.py and bin/db_simplify.py were added. These are undocumented and untested and only meant as examples to help diagnose problems.


11-24
Advance dock_dismod_at.sh version 20201122. This should speed up the build process. It requires one to re-build dismod_at.mixed and dismod_at.image.

11-22
Advance to cppad_mixed-20201124. This speeds up the build process when multiple processors are available. It also fixes a problem in cppad_mixed.pc and thereby removes the need to specify a prefix for each of the special requirements .

11-23
  1. Advance to cppad_mixed and dock_dismod_at.sh version 20201122. This simplifies the build process because the script for building and installing cppad_mixed, and its dependencies, come from cppad_mixed. In addition, it speed up the build process when multiple processors are available.
  2. API change for dock_dismod_at.sh . To be specific, there are now three dismod_at docker images: dismod_at.base , dismod_at.mixed , dismod_at.image . You must rebuild all of these images.


10-10
Improve the wording in the documentation of std in the prior table.

08-31
Advance to cppad_mixed and dock_dismod_at.sh version 20200831. This fixes a bug in the calculation of the first derivative of the fixed effects objective. This bug only occurred when ldlt_cholmod was yes in bin/install_cppad_mixed.sh and the factorization of the random effects Hessian had a permutation that did not act as its own inverse. To use the new dock_dismod_at.sh you will have to build a new dismod_at.image .

08-23
  1. Fix location of upstream source in bin/install_eigen.
  2. Patch Ipopt's source file IpIpoptApplication.cpp; see ipopt issue 393 .


08-20
Change bin/install_ipopt.sh to directly download and configure the necessary third party software (not use coinbrew).

08-16
Improve the debug build_type so that ipopt and all its libraries also get built for debugging.

07-02
  1. Fix a problem with the change on the previous day (see 07-01 below). To be specific, the tables for the Hessian with respect to the fixed and random effect were not being created in some error cases that should create them (to help diagnose the problem). This required advancing to cppad_mixed-20200702.
  2. Advance the dock_dismod_at.sh dismod_at_version to 20200702. This will require building a new dismod_at.image .


07-01
Dismod_at will exit with an error when the sample_table is not created by the sample asymptotic command. If this is because the fixed effects information matrix is not positive definite, the hes_fixed_table will contain the information matrix

06-30
Advance to cppad_mixed-20200630. This fixes a problem where the sample_table was invalid after the sample asymptotic command when an information matrix that is not positive definite. Now, in addition to the warning about the information matrix, the sample table does not exist after the command when this happens.

06-24
Advance to the dock_dismod_at.sh dismod_at_version to 20200620 This will require building a new dismod_at.base and then dismod_at.image .

06-20
The variables argument was added to the sample command. This enables one to get statistics for fitting just the fixed effects. It is not backward compatible because you will have to include the variables argument in your sample commands. To get the old behavior, set variables to both.

06-08
Advance to ipopt-3.13.2, cppad-20200606, cppad_mixed-20200608.

05-30
Advance to cppad_mixed-20200530 (more work on the adaptive derivative checker).

05-29
  1. A check was added for the case where there are more constraints on the random effects than there are random effects.
  2. Add the user_subgroup_mulcov.py example. This combines fixed and random effects for both parent/child and group/subgroup and for the same rate.
  3. The standard deviation was removed from the prior for some uniform priors in user_bilevel_random.py (it was not being used).
  4. Advance to cppad_mixed-20200529. This adds a missing error report when the Hessian w.r.t. the random effects is not positive definite at the starting variable values and quasi_fixed is false.


05-27
Advance to cppad_mixed-20200527. This should improve the output generated by the derivative_test trace-adaptive option.

05-21
The user_covid_19.py example was added.

05-15
Add another restriction on the group_name . To be specific, the only case were a group name can also be a subgroup name is when the group has only one subgroup and the same name is used for both. In addition, more subgroup table error checking (that used to be only in the debug version) is done in the release version of dismod_at.

03-28
  1. Add simplification to the scaling of the asymptotic statistics in the case were we have optimal fixed effects .
  2. Advance to cppad_mixed-20200328. This a bug in the release version during asymptotic sampling when there are no random effects.


03-27
  1. Add the user_one_function.py which demonstrates how to fit an arbitrary function and ignore the dismod_at differential equation .
  2. Add the no_ode option to the rate case in the option table. This was necessary to simplify the one function example mentioned above.
  3. The log scaling discussion for standard deviations in the prior table was edited to make it clearer that difference standard deviations are not transformed.
  4. The discussion of log transformed standard deviations sigma was edited to make it clearer why it is different for data than it is for priors.
  5. Remove incorrect sentence about the simulated random effects depending on the simulated fixed effects in the asymptotic method.


03-26
The user_posterior.py example was moved to user_sample_asy_sim.py and it was extended to include the asymptotic sampling method.

03-25
  1. Advance to cppad_mixed-20200325. This fixed an assert in the debug version during asymptotic sampling when there are no random effects.
  2. If during asymptotic sampling the Hessian of the fixed or random effects object is not positive definite, print a warning and do not create a sample table (instead of filling the sample table with nulls); see no sample table . This fixed a bug where the db2csv_command tried to average null values.
  3. The sample command asymptotic method uses a normal distribution, but the scaled fixed effects variables are normal in the scaled space; see fixed effects distribution
  4. Add a test of scaling fixed effects to the hes_fixed_math.py example.
  5. Advance to the dock_dismod_at.sh dismod_at_version to 20200325. This will require building a new dismod_at.image .


03-24
  1. The Hessian tables are now guaranteed to be in row major order; see representation for the hes_fixed and hes_random tables.
  2. The values in hes_fixed_table are now in scaled space; see scaling fixed effects .
  3. Create separate tables, in the data flow section, for the command output tables and command extra input tables .
  4. Add the hes_fixed.csv and hes_random.csv output files to the db2csv command.
  5. Check the hes_random.csv file at the end of the user_hes_random.py example.


03-23
  1. Update to using version cppad_mixed-20200319. This enables computation of the Hessian of the random effects objective.
  2. Add the hes_random_table as an output during an asymptotic sample command. Add the user_hes_random.py example use of the hes_random table.


03-22
  1. Add the hes_fixed table as an extra output when computing asymptotic samples.
  2. The user_asymptotic_math.py example was renamed to user_hes_fixed_math.py because it checks the hes_fixed_table .
  3. Add a connecting sentence about computation before the equations for the derivatives of @(@ \hat{u}_i ( \theta ) @)@ and @(@ g_i ( \theta ) @)@.
  4. The asymptotic sampling of the random effects was changed to use the a fixed Hessian w.r.t the random effects for all the samples. Trying to optimize the random effects given the simulated fixed effects, without changing the data, often fails and does not make sense.


03-21
Add the user_hes_fixed_math.py example.

03-19
Update to using version cppad_mixed-20200319. This fixes some asserts during the asymptotic sampling method in the debug build.

03-18
  1. Advance the dock_dismod_at.sh dismod_at_version to 20200318. This will require building a new dismod_at.image .
  2. Add the simulate_index argument to the asymptotic version of the sample command. This fixed a bug where the asymptotic statistics for a fit using simulated data was using meas_value when it should have been using data_sim_value .
  3. Fix a bug in the sampling of the random effect using the asymptotic method.
  4. The user_speed.py example was improved. More documentation was added, it has a realistic parent and child grids, and it works with both quasi-Newton and Newton method, and it includes an example / test of the asymptotic statistics.


03-15
  1. Advance the dock_dismod_at.sh dismod_at_version to 20200315. This will require building a new dismod_at.image .
  2. The sample command asymptotic method was changed to ignore all the constraints except for model_variables that have their upper and lower limits equal. This should speed up the computation of the asymptotic statics. It also provides a way to approximation the Hessian of the fixed effect objective (as the inverse of the sample covariance). Note that the simulated samples may be outside the bounds for the corresponding variables; see bounds . Also note that the predict_command censors the samples to be within their limits before computing its predictions.
  3. Some minor corrections were made to the discussion of the user_sample_asy.py example; e.g. @(@ \iota_c @)@ and @(@ \iota_m @)@ where changed to @(@ \iota_n @)@. In addition, the problem parameters were include in the documentation.


03-09
  1. There was a bug in the smoothing when there was a upper or lower bound in the prior for dage or dtime prior for a fixed effect that was scaled; see scaling fixed effects . This has been fixed.
  2. Advance the dock_dismod_at.sh dismod_at_version to 20200309. This will require building a new dismod_at.image .


03-06
The jump_at_age.py example has been improved to include more discussion and the flexibility to change its parameters .

02-29
  1. Add the user_residual.py example.
  2. If the user specified the parent_node_id in the option table, the corresponding parent node name is placed in the option.csv .


02-28
Fix the formula for the age smoothing A^s , and the time smoothing T^s . To be specific, the forward index comes before the current index; e.g., @(@ D(v_{i,j}, v_{i+1,j} \ldots ) @)@ was replaced by @(@ D(v_{i+1,j}, v_{i,j} \ldots ) @)@.

02-26
  1. Add an immunity wish list item.
  2. Discuss the difference between the weighted residual in the log scaled value and the log scaled difference cases.
  3. Discuss the difference in the definition of the @(@ \sigma @)@ in the data versus prior cases.


02-20
Advance the dock_dismod_at.sh dismod_at_version to 20200218. This will require building a new dismod_at.image .

02-18
The dismod_at program would crash if the subgroup_id in the data_table or avgint_table as null. This has been fixed by detecting this case and printing an error message.

02-14
The description of the hold_out data has been improved.

02-13
  1. The age_id and time_id values in the smooth_grid_table and weight_grid_table were not being checked for validity. This has been changed so that the user gets an informative error message when they are outside the valid limits.
  2. Advance the dock_dismod_at.sh dismod_at_version to 20200213. This will require building a new dismod_at.image .


01-31
A simulating priors entry was added to the wish list.

01-16
Add an optimizer warm start wish list item.

01-08
  1. Add documentation for the arguments database and command under running a new container.
  2. Advance the dock_dismod_at.sh dismod_at_version to 2020010. of dock_dismod_at.sh. This will require building a new dismod_at.image .


01-07
Fix the following compile error when NDEBUG is not defined:
   .../devel/table/get_subgroup_table.cpp:128:31: error:
   'subgroup_id' was not declared in this scope
Input File: omh/whats_new/2020.omh