Printing profile on sig kill. (#44)

* added skeleton for terminal logger

* done skeleton terminal writer logic... todo: actual calls to the write classes

* moved lambda to top function

* added logic to support timing

* prettier printing

* prettier printing

* lint

* separated out timing and other stats variables. printing in dict format

* keeping logging history and wiping rest

* updated logic to calculate total times instead of average times

* made stats tracker configurable

* moved stats tracker and profiler to own class file; implemented basic functionality for profiler

* lint removing duplicate function

* update readme; moved around config variables

* profiler prints on sig kill
This commit is contained in:
evonneng authored and GitHub committed 2022-04-29 11:10:34 -07:00
1 parent 47691eb291
commit 86b1011ae1
8 files changed
+46 -24

No files matched your search

+14 -3
View File
@@ -1,7 +1,8 @@
# MattPort
# Setup the environment
# Quickstart
## Installation: Setup the environment
```
# Clone the repo
@@ -19,7 +20,7 @@ python setup.py develop
pytest tests
```
# Getting the data
## Getting the data
Download the original NeRF dataset and put it in the following format.
@@ -30,7 +31,7 @@ Download the original NeRF dataset and put it in the following format.
- lego/
```
# Training a model
## Training a model
```
# Run with the default config
@@ -39,6 +40,16 @@ python scripts/run_train.py
python scripts/run_train.py machine_config.num_gpus=1
```
# Logging and profiling features
We provide many logging functionalities for timing and/or tracking losses during training. All of these loggers are configurable via `configs/logging_configs.yml`
1. **Writer**: Logs losses and generated images during training to a specified output stream. Specify the type of writer (Tensorboard, local directory, Weights and Biases), and how often to log in the config.
2. **StatsTracker**: Computes select statistics and prints to the terminal for local debugging. Specify the kinds of statistics to be logged, how often to log, and the maximum history buffer size in the config.
3. **Profiler**: Computes the average total time of execution for any function with the `@profiler.time_function` decorator. Prints out the full profile at the termination of training or the program.
# Setting up Jupyter
```