Resource allocation colormap #3453
Conversation
There was a problem hiding this comment.
Do we need this file? Can it be deleted?
There was a problem hiding this comment.
We can delete this file.
There was a problem hiding this comment.
OK, thank you; then please rm.
We could do that. I think the best approach would be to have a function that would convert Python expression (e.g. np.log(x) * k...) into a string and upload it to DB similar to resource allocation table. |
|
To be honest I think a direct insert would be easier - at the end, only admins with DB access can add them. |
qiita_db/util.py
Outdated
|
|
||
| return fig, axs | ||
|
|
||
|
|
||
| def _retrieve_equations(): |
There was a problem hiding this comment.
Can you make this public by removing the _?
qiita_db/util.py
Outdated
| for models in res: | ||
| if 'mem' in models[1]: | ||
| memory_models[models[1]] = lambda x, k, a, b: eval(models[2]) | ||
| else: | ||
| time_models[models[1]] = lambda x, k, a, b: eval(models[2]) | ||
| return (memory_models, time_models) |
There was a problem hiding this comment.
This loop can be outside the TRN, right?
qiita_db/util.py
Outdated
| + np.log(x) * k) | ||
|
|
||
| MODELS_TIME = [time_model1, time_model2, time_model3, time_model4] | ||
|
|
||
|
|
||
| def get_model_name(model): |
There was a problem hiding this comment.
This method can really be replaced by _retrieve_equations, no? I mean, I don't understand why we need to have this here vs. using _retrieve_equations.
There was a problem hiding this comment.
OK, thank you; then please rm.
| @@ -62,3 +62,11 @@ CREATE INDEX IF NOT EXISTS processing_job_command_parameters_payload ON qiita.pr | |||
| -- Addding contraints for the slurm_reservation column | |||
| ALTER TABLE qiita.analysis DROP CONSTRAINT IF EXISTS analysis_slurm_reservation_valid_chars; | |||
| ALTER TABLE qiita.analysis ADD CONSTRAINT analysis_slurm_reservation_valid_chars CHECK ( slurm_reservation ~ '^[a-zA-Z0-9_]*$' ); | |||
|
|
|||
There was a problem hiding this comment.
Note that we did a release this morning so these lines need to be moved to 94.sql.
| @@ -0,0 +1,10 @@ | |||
| INSERT INTO qiita.allocation_equations(equation_name, expression) | |||
There was a problem hiding this comment.
Same as the other sql: these lines need to be moved to 94.sql.
| """) | ||
| self.assertEqual(failures, 1, "Number of failures must be 1") | ||
| self.assertEqual(failures, 0, "Number of failures must be 0") |
There was a problem hiding this comment.
Just wondering why this changed from 1 to 0?
There was a problem hiding this comment.
Now the code chooses 4th model instead of 1st, which has 0 failures.
qiita_db/util.py
Outdated
| for models in res: | ||
| model_name = "Unknown model" | ||
| if models[1] == 'mem_model1': | ||
| model_name = "k * log(x) + x * a + b" | ||
| elif models[1] == 'mem_model2': | ||
| model_name = "k * log(x) + b * log(x)^2 + a" | ||
| elif models[1] == 'mem_model3': | ||
| model_name = "k * log(x) + b * log(x)^2 + a * log(x)^3" | ||
| elif models[1] == 'mem_model4': | ||
| model_name = "k * log(x) + b * log(x)^2 + a * log(x)^2.5" | ||
| elif models[1] == 'time_model1': | ||
| model_name = "a + b + log(x) * k" | ||
| elif models[1] == 'time_model2': | ||
| model_name = "a + b * x + log(x) * k" | ||
| elif models[1] == 'time_model3': | ||
| model_name = "a + b * log(x)^2 + log(x) * k" | ||
| elif models[1] == 'time_model4': | ||
| model_name = "a * log(x)^3 + b * log(x)^2 + log(x) * k" | ||
| if 'mem' in models[1]: | ||
| memory_models[models[1]] = { | ||
| "equation_name": model_name, | ||
| "equation": lambda x, k, a, b: eval(models[2]) | ||
| } |
There was a problem hiding this comment.
Why do we need this? I mean the model_name is the expression column in the database, no?
There was a problem hiding this comment.
We have this for readability on the frontend. It was hard to read "np.log10(x)...". Maybe there's a way to automatically convert those from lambda function to a readable equation?
There was a problem hiding this comment.
I see, I think leaving as np.log or doing just log in the database will be fine. Note that if you replace np.log for log in the database you would need to change the import numpy as np to from numpy import log.
There was a problem hiding this comment.
We previously also talked about adding parenthesis to ensure correct order of operations between systems, which makes the readability worse. Should we still keep the parenthesis or should we assume the regular order of operations? For example:
(k * (np.log(x))) + (b * ((np.log(x))**2)) + (a * ((np.log(x))**3))
Could just be k * log(x) + b * log(x)**2 + a * log(x)**3
There was a problem hiding this comment.
I like the extra parenthesis so we don't assume the parenthesis order.
No description provided.