Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove standard topmodel output when running from ngen #41

Merged
merged 62 commits into from
Mar 29, 2024
Merged

Remove standard topmodel output when running from ngen #41

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
11b32be
testing fork
Ben-Choat Jul 25, 2023
bfea366
removing test file
Ben-Choat Jul 25, 2023
fe7cd9b
Failed attempt at implementing calibratable params
Ben-Choat Aug 9, 2023
bdc39b0
backup/scratch working files to be removed later
Ben-Choat Aug 9, 2023
4d40b80
fixing an accidental comment included with this file
Ben-Choat Aug 9, 2023
9c0745c
remove nstep dt from config and set to 1 in else block initalize - fi…
Ben-Choat Aug 9, 2023
122ae65
refactoring init to smaller functions
Ben-Choat Aug 15, 2023
e29eed6
refactored init() into smaller functions
Ben-Choat Aug 16, 2023
d8c7221
refactored init() into smaller functions
Ben-Choat Aug 16, 2023
bbfa55c
refactored init() to smaller functions
Ben-Choat Aug 16, 2023
60dbc77
refactoring init() to smaller functions - update comments
Ben-Choat Aug 16, 2023
53b0419
refactored init() to smaller functions - added missing \+
Ben-Choat Aug 16, 2023
76a8988
removing file that was not intended to push originally, and was only …
Ben-Choat Aug 29, 2023
840028c
adding refactored init() function to update calibratable parameters
Ben-Choat Aug 29, 2023
5607002
better organizing and adding comments in code
Ben-Choat Aug 29, 2023
59ae02d
removing folder that was not not originally intended to push
Ben-Choat Aug 29, 2023
8264d68
updating commenting and including new init functions in bmi_topmodel.c
Ben-Choat Aug 30, 2023
7756c24
edited init() prototype tp match change of order of input variables i…
Ben-Choat Aug 30, 2023
aca60e7
changed order of init() input variables so outputs come last. Also up…
Ben-Choat Aug 30, 2023
54675ee
imporved comments and organization
Ben-Choat Aug 31, 2023
558f90a
Toward updating calibratable params with refactored functions
Ben-Choat Aug 31, 2023
7c1f590
added lines to include chv, rv as paramters and include the prototype…
Ben-Choat Aug 31, 2023
9eb924f
updating comments for clarity
Ben-Choat Sep 5, 2023
9eb91bf
actively working on using refactored init() functions to update calib…
Ben-Choat Sep 5, 2023
116d98c
added chv & rv to model data structure. Updated order of init() proto…
Ben-Choat Sep 6, 2023
5529661
updated Set_value function to incorporate calibratable parameters. St…
Ben-Choat Sep 6, 2023
609fa10
updated order of init() input variables. Added some print statements …
Ben-Choat Sep 6, 2023
55298a8
updated output messages related to calibratable parameters
Ben-Choat Sep 6, 2023
fb822d2
made small edits to Set_value - removed uneeded calibratable parameters
Ben-Choat Sep 6, 2023
14f875d
updated code related to calibrated parameters in Set_value().
Ben-Choat Sep 7, 2023
379f352
edited order of refactored functions to be more logical. added print …
Ben-Choat Sep 7, 2023
8c5d597
added a few print lines for troubleshooting. Unexpected behavior rela…
Ben-Choat Sep 7, 2023
f6a8ca8
removed print statements no longer needed/relevant
Ben-Choat Sep 8, 2023
aca081d
removed/edited comments no longer needed
Ben-Choat Sep 8, 2023
1bbd41d
small updates to comments
Ben-Choat Sep 11, 2023
c524a3a
updating some comments
Ben-Choat Sep 11, 2023
6e58fd1
updated comments
Ben-Choat Sep 11, 2023
1b06db4
removed uneeded comments
Ben-Choat Sep 11, 2023
cfbdec5
corrected a typo in comments
Ben-Choat Sep 11, 2023
3b25e5c
add helper function to shift Q array by one
hellkite500 Sep 20, 2023
98c2a1c
properly account for time delayed flow accumulation in Q
hellkite500 Sep 20, 2023
c81f69d
use dynamic tch array in convert_dist_to_histords
hellkite500 Sep 20, 2023
00db5ea
use dynamic tch array in calc_time_delay_histogram
hellkite500 Sep 20, 2023
dd56d13
simplify ordinate area adjustment and validate oridnates sum to 1
hellkite500 Sep 20, 2023
b625c15
ensure Q is dynamically initialized
hellkite500 Sep 20, 2023
0042550
Add macros for model limit warnings
hellkite500 Sep 20, 2023
da5b500
Remove hard coded program limits
hellkite500 Sep 20, 2023
92c6017
fix logical operator
hellkite500 Sep 20, 2023
8bd02b6
small combine loop optimization
hellkite500 Sep 20, 2023
625e351
add debug and sanitize flags to unit test build
hellkite500 Sep 20, 2023
fc066cb
fix bug in printing value from pointer
hellkite500 Sep 20, 2023
2ffd6e4
comment bmi Q initialization
hellkite500 Sep 20, 2023
76c3b3f
temporarly 'fix' standalone mode allocation of Q in driver
hellkite500 Sep 20, 2023
1978a88
Corrected amount Q is shifted by at each timestep to include num_dela…
Ben-Choat Sep 21, 2023
38f3a95
Added stand_alone to init() and init_discharge, and edited to allocat…
Ben-Choat Sep 22, 2023
8c7ad8c
fixed error - szq and sum_histogram_ordinates were reported incorrect…
Ben-Choat Sep 25, 2023
9f3b0e0
cleaned up some indentation/formatting
Ben-Choat Sep 26, 2023
0c346a4
Edited so standard topmodel output files are only written when in sta…
Ben-Choat Sep 29, 2023
51959a2
added if statements so info about calibratable parameters is only pri…
Ben-Choat Sep 29, 2023
aec96a6
added stand_alone as input to water balance to fix seg fault when in …
Ben-Choat Sep 29, 2023
9330aec
Merge remote-tracking branch 'origin/master' into remove-topmodel-out…
Ben-Choat Jan 15, 2024
7dfb805
merging w/master
Ben-Choat Jan 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions include/topmodel.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,7 @@ extern void init(FILE *in_param_fptr, FILE *output_fptr, char *subcat, int stand
double **stor_unsat_zone, double **deficit_local,
double **deficit_root_zone,double *szq,double **Q,
double *sbar, double *bal);



extern void inputs(FILE *input_fptr, int *nstep, double *dt, double **rain,
double **pe, double **Qobs, double **Q, double **contrib_area);

Expand All @@ -73,7 +72,7 @@ extern void topmod(FILE *output_fptr, int nstep, int num_topodex_values,
extern void tread(FILE *subcat_fptr,FILE *output_fptr,char *subcat,
int *num_topodex_values,int *num_channels,double *area,
double **dist_area_lnaotb, double **lnaotb, int yes_print_output,
double **cum_dist_area_with_dist, double *tl,
int stand_alone, double **cum_dist_area_with_dist, double *tl,
double **dist_from_outlet);

extern void expinf(int irof, int it, int rint, double *df, double *cumf,
Expand All @@ -83,7 +82,7 @@ extern void results(FILE *output_fptr, FILE *out_hyd_fptr,
int nstep, double *Qobs, double *Q,
int yes_print_output);

extern void water_balance(FILE *output_fptr, int yes_print_output,
extern void water_balance(FILE *output_fptr, int yes_print_output, int stand_alone,
char *subcat, double *bal, double *sbar, double *sump,
double *sumae, double *sumq, double *sumrz, double *sumuz);

Expand Down
21 changes: 8 additions & 13 deletions src/bmi_topmodel.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,7 @@ int read_init_config(const char* config_file, topmodel_model* model) {
fscanf(model->subcat_fptr,"%d %d %d",&model->num_sub_catchments,&model->imap,&model->yes_print_output);

// Attempt to read the output file names only if printing to file
if(model->yes_print_output == TRUE){
if(model->yes_print_output == TRUE && model->stand_alone == TRUE){
if((model->output_fptr=fopen(output_fname,"w"))==NULL){
printf("Can't open output file named %s\n",output_fname);
exit(-9);
Expand Down Expand Up @@ -298,7 +298,7 @@ int init_config(const char* config_file, topmodel_model* model)
}

tread(model->subcat_fptr,model->output_fptr,model->subcat,&model->num_topodex_values,&model->num_channels,
&model->area,&model->dist_area_lnaotb,&model->lnaotb,model->yes_print_output,
&model->area,&model->dist_area_lnaotb,&model->lnaotb,model->yes_print_output,model->stand_alone,
&model->cum_dist_area_with_dist,&model->tl,&model->dist_from_outlet);

fclose(model->subcat_fptr);
Expand Down Expand Up @@ -434,10 +434,6 @@ static int Update (Bmi *self)
&topmodel->sump,&topmodel->sumae,&topmodel->sumq,&topmodel->sumrz,&topmodel->sumuz,
&topmodel->quz, &topmodel->qb, &topmodel->qof, &topmodel->p, &topmodel->ep );

if ((topmodel->stand_alone == FALSE) && (topmodel->yes_print_output == TRUE)){
fprintf(topmodel->out_hyd_fptr,"%d %lf %lf\n",topmodel->current_time_step,topmodel->Qobs[1],topmodel->Q[1]);
}

return BMI_SUCCESS;
}

Expand Down Expand Up @@ -476,14 +472,12 @@ static int Finalize (Bmi *self)
if (self){
topmodel_model* model = (topmodel_model *)(self->data);

if (model->yes_print_output == TRUE || TOPMODEL_DEBUG >= 1){

water_balance(model->output_fptr, model->yes_print_output,
if (model->yes_print_output == TRUE && model->stand_alone == TRUE || TOPMODEL_DEBUG >= 1){
water_balance(model->output_fptr, model->yes_print_output, model->stand_alone,
model->subcat,&model->bal, &model->sbar, &model->sump,
&model->sumae, &model->sumq, &model->sumrz, &model->sumuz);

// this is technically needed, yes
if(model->yes_print_output==TRUE){
if(model->yes_print_output==TRUE && model->stand_alone==TRUE){
fprintf(model->output_fptr,"Maximum contributing area %12.5lf\n",model->max_contrib_area);
}

Expand Down Expand Up @@ -527,7 +521,7 @@ static int Finalize (Bmi *self)
free(model->dist_from_outlet);

// Close output files only if opened in first place
if(model->yes_print_output == TRUE){
if(model->yes_print_output == TRUE && model->stand_alone == TRUE){ // && model->stand_alone == TRUE){
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spurious comment

fclose(model->output_fptr);
fclose(model->out_hyd_fptr);
}
Expand Down Expand Up @@ -996,6 +990,7 @@ static int Set_value (Bmi *self, const char *name, void *array)
// assign self->data to topmodel pointer
topmodel = (topmodel_model *) self->data;

#if TOPMODEL_DEBUG >= 1
printf("\n\n\nAT LEAST ONE OF THE FOLLOWING CALIBRATABLE PARAMETERS "
"WAS PROVIDED IN THE REALIZATION.JSON FILE!\n");

Expand All @@ -1013,7 +1008,7 @@ static int Set_value (Bmi *self, const char *name, void *array)
printf("szm = %f\n", topmodel->szm);
printf("sr0 = %f\n", topmodel->sr0);
printf("t0 = %f\n\n\n\n", topmodel->t0);

#endif
}

// UPDATE APPROPRIATE PARAMETERS
Expand Down
56 changes: 28 additions & 28 deletions src/topmodel.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ max_contrib_area=0.0;
/* Note: original source code starts for-loop over nstep here
Also recall that bmi adaptation removes submatchment loop
which lived in topmod9502.c main() function */
if(yes_print_output==TRUE && current_time_step==1)
if(yes_print_output==TRUE && current_time_step==1 && stand_alone==TRUE)
{
fprintf(output_fptr,
"it p ep q(it) quz q sbar qof\n");
Expand Down Expand Up @@ -340,7 +340,7 @@ for(ir=1;ir<=num_time_delay_histo_ords;ir++)
//Add current time flow to mass balance variable
*sumq += Q[it];
/* BMI Adaption: replace nstep with current_time_step */
if(yes_print_output==TRUE && in<=current_time_step)
if(yes_print_output==TRUE && in<=current_time_step && stand_alone==TRUE)
{
fprintf(output_fptr,"%d %6.4e %6.4e %6.4e %6.4e %6.4e %6.4e %6.4e\n",
current_time_step, (*p), (*ep), Q[it], (*quz), (*qb), (*sbar), (*qof));
Expand Down Expand Up @@ -368,7 +368,7 @@ return;

}

extern void water_balance(FILE *output_fptr, int yes_print_output,
extern void water_balance(FILE *output_fptr, int yes_print_output, int stand_alone,
char *subcat,double *bal, double *sbar, double *sump,
double *sumae, double *sumq, double *sumrz, double *sumuz)
{
Expand All @@ -383,8 +383,8 @@ extern void water_balance(FILE *output_fptr, int yes_print_output,
(*sump),(*sumae),(*sumq),(*sumrz),(*sumuz),(*sbar),(*bal));
#endif

if (yes_print_output==TRUE)
{
if (yes_print_output==TRUE && stand_alone==TRUE)
{
fprintf(output_fptr,"\nWater Balance for Subcatchment: %s\n",subcat);
fprintf(output_fptr,
" SUMP SUMAE SUMQ SUMRZ SUMUZ SBAR BAL\n");
Expand Down Expand Up @@ -421,8 +421,7 @@ d_alloc(Qobs,(*nstep));
//NJF This is dangerous, there is no validation between nstep and
//number of historgram ordinates, which is used to iterate Q in later steps
//so this could easily overflow if nstep is smaller than historgram ords
d_alloc(Q,(*nstep)); //NJF TODO validate that this works correctly
//When used in "stand alone" mode
d_alloc(Q,(*nstep));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this ever validated? I don't believe I went through that code path in detail throughout the dyamic alloc refactoring. Might want to leave this comment until we are sure there isn't possible issues here in stand alone mode?

Copy link
Contributor Author

@Ben-Choat Ben-Choat Jan 17, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am able to run in stand alone mode and with NGEN. Output files topmod.out and hyd.out are produced and seem correct. Not sure if further validation is needed.

On a tangential note, to run in stand_alone I am using:

#!/bin/bash
# -lm links math library
gcc ./extern/topmodel/topmodel/src/main.c \
        ./extern/topmodel/topmodel/src/bmi_topmodel.c \
        ./extern/topmodel/topmodel/src/topmodel.c -o\
        run_topmodel_bmi -lm

./run_topmodel_bmi

When I view the info in docs/STAND_ALONE.md, I do not see these instructions anywhere. If this is the expected way to execute in stand_alone, it may be worth adding those instructions there.

If it is not the expected way to execute in stand_alone mode, feel free to let me know!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ben-Choat - The install doc gives instructions on standalone via cmake. It probably wouldn't be a bad idea to add a link in the STAND_ALONE.md. Thanks for pointing this out.

d_alloc(contrib_area,(*nstep));

//---------------------------------------
Expand All @@ -441,7 +440,7 @@ return;
extern void tread(FILE *subcat_fptr,FILE *output_fptr,char *subcat,
int *num_topodex_values,int *num_channels,double *area,
double **dist_area_lnaotb,double **lnaotb, int yes_print_output,
double **cum_dist_area_with_dist,double *tl,
int stand_alone, double **cum_dist_area_with_dist,double *tl,
double **dist_from_outlet)
{
/**************************************************************
Expand All @@ -458,7 +457,7 @@ int j;
fgets(subcat,256,subcat_fptr); /* do twice to read in line-feed */
fgets(subcat,256,subcat_fptr);

if (yes_print_output == TRUE)
if (yes_print_output == TRUE && stand_alone == TRUE)
{
fprintf(output_fptr,"Subcatchment : %s\n",subcat);
}
Expand Down Expand Up @@ -540,7 +539,7 @@ for(j=1;j<=(*num_channels);j++)
/* dist_from_outlet[1] is distance from subcatchment outlet */
/* cum_dist_area_with_dist[1] = 0. */

if(yes_print_output==TRUE)
if(yes_print_output==TRUE && stand_alone==TRUE)
{
fprintf(output_fptr,"TL = %8.2lf\n",(*tl));
fprintf(output_fptr,"SUMAC = %8.2lf\n",sumac);
Expand All @@ -555,8 +554,9 @@ return;
* to enable calibratable parameters to be updated.
* BChoat 2023/08/29
*/
/**


/**
* Function to convert distance/area form to time delay histogram ordinates
*
* Converts parameters to m/time step DT
Expand Down Expand Up @@ -929,7 +929,6 @@ double tch[num_channels+1]; //+1 to maintain 1 based indexing used by other rout
double sumar;
int ir;


/* read in run parameters */
fgets(subcat,256,in_param_fptr);

Expand All @@ -938,21 +937,22 @@ printf("subcat: %s\n", subcat);
fscanf(in_param_fptr,"%lf %lf %lf %lf %lf %lf %lf %lf %d %lf %lf %lf",
szm,t0,td,chv,rv,srmax,Q0,sr0,infex,xk0,hf,dth);


printf("\n\nCalibratable parameters from params*.dat:\n");

printf("\nET and recharge:\n");
printf("srmax = %f\n", *srmax);
printf("td = %f\n", *td);

printf("\nDischarge:\n");
printf("chv = %f\n", *chv);
printf("rv = %f\n", *rv);

printf("\nWater balance:\n");
printf("szm = %f\n", *szm);
printf("sr0 = %f\n", *sr0);
printf("t0 = %f\n\n", *t0);
#if TOPMODEL_DEBUG >= 1
printf("\n\nCalibratable parameters from params*.dat:\n");

printf("\nET and recharge:\n");
printf("srmax = %f\n", *srmax);
printf("td = %f\n", *td);

printf("\nDischarge:\n");
printf("chv = %f\n", *chv);
printf("rv = %f\n", *rv);

printf("\nWater balance:\n");
printf("szm = %f\n", *szm);
printf("sr0 = %f\n", *sr0);
printf("t0 = %f\n\n", *t0);
#endif

//NJF num_channels is the value provided (SHOULD COME FROM TREAD)
//Convert distance/area form to time delay histogram ordinates
Expand All @@ -976,7 +976,7 @@ init_water_balance(num_topodex_values,
stor_unsat_zone, szq,
deficit_local, deficit_root_zone, sbar, bal);

if(yes_print_output==TRUE)
if(yes_print_output==TRUE && stand_alone == TRUE)
{

// calculate sum of hist ords for printing to output file
Expand Down