This tutorial by Kronos focuses on the specific task in the title. Kronos is a financial K-line modeling project; the forecast output cannot be regarded as investment advice. Data leakage, re-righting methods, transaction costs and out-of-sample testing will significantly change the backtest results.
Below is a demonstration using a test environment and revocable data. When accounts, keys, machine control, investments or emergency communications are involved, manual confirmation must be done.
First clarify whether the task is to reproduce or research
This section only deals with “first clarify whether the task is to reproduce or research”. Save the current state first and then run the minimal test. Do not change multiple conditions at the same time.
|
|
For Kronos, this boundary must be combined when judging the results: Kronos is a financial K-line modeling project; the forecast output cannot be used as investment advice, and data leakage, re-righting methods, transaction costs and out-of-sample testing will significantly change the backtest results.
If the results are not repeatable, record the version, input, and logs, then narrow the scope of the test.
Fixed Python, CUDA and dependency versions
It is recommended to execute in the following order:
- Check the actual version and source.
- Use a test account, test directory, or test device.
- Perform an undoable action.
- Save the output, exit code, or screenshot.
- Revert the changes and re-verify.
|
|
The completion standard is to “fix Python, CUDA and dependent versions” to obtain verifiable results, rather than the command just not reporting an error.
Create an environment isolated from other projects
| Check items | Passing criteria | Stop signal |
|---|---|---|
| Create an environment isolated from other projects | Input, output and permission scope are clear | Tools automatically expand the scope of operations |
| Source | Can return to the official warehouse or original data | Rely on unknown mirrors or second-hand conclusions |
| Security | No secret value in log | Token, cookie or private data present |
| Recovery | Ability to return to pre-execution state | No backup or undo methods |
|
|
When a stop signal appears, resume first and do not continue to the next step.
Model weight and code version must match
This section only deals with “model weights and code versions must match”. Save the current state first and then run the minimal test. Do not change multiple conditions at the same time.
|
|
For Kronos, this boundary must be combined when judging the results: Kronos is a financial K-line modeling project; the forecast output cannot be used as investment advice, and data leakage, re-righting methods, transaction costs and out-of-sample testing will significantly change the backtest results.
If the results are not repeatable, record the version, input, and logs, then narrow the scope of the test.
The order of K-line fields cannot be guessed.
It is recommended to execute in the following order:
- Check the actual version and source.
- Use a test account, test directory, or test device.
- Perform an undoable action.
- Save the output, exit code, or screenshot.
- Revert the changes and re-verify.
|
|
The completion standard is that “the order of the K-line fields cannot be guessed” to obtain a verifiable result, rather than that the command happens to not report an error.
Time zone, suspension and missing values are processed first
| Check items | Passing criteria | Stop signal |
|---|---|---|
| Time zones, suspensions and missing values are processed first | Input, output and authority scope are clear | Tools automatically expand the scope of operations |
| Source | Can return to the official warehouse or original data | Rely on unknown mirrors or second-hand conclusions |
| Security | No secret value in log | Token, cookie or private data present |
| Recovery | Ability to return to pre-execution state | No backup or undo methods |
|
|
When a stop signal appears, resume first and do not continue to the next step.
What will change before and after reinstatement?
This section only deals with “what will change before and after reinstatement”. Save the current state first and then run the minimal test. Do not change multiple conditions at the same time.
|
|
For Kronos, this boundary must be combined when judging the results: Kronos is a financial K-line modeling project; the forecast output cannot be used as investment advice, and data leakage, re-righting methods, transaction costs and out-of-sample testing will significantly change the backtest results.
If the results are not repeatable, record the version, input, and logs, then narrow the scope of the test.
Only run the official minimum sample for the first time
It is recommended to execute in the following order:
- Check the actual version and source.
- Use a test account, test directory, or test device.
- Perform an undoable action.
- Save the output, exit code, or screenshot.
- Revert the changes and re-verify.
|
|
The completion standard is to “run only the official minimum sample for the first time” to obtain verifiable results, rather than the command just not reporting an error.
If there is insufficient video memory, shorten the sequence and batch first.
| Check items | Passing criteria | Stop signal |
|---|---|---|
| If there is insufficient video memory, shorten the sequence and batch first | Clear input, output and permission ranges | The tool automatically expands the scope of operations |
| Source | Can return to the official warehouse or original data | Rely on unknown mirrors or second-hand conclusions |
| Security | No secret value in log | Token, cookie or private data present |
| Recovery | Ability to return to pre-execution state | No backup or undo methods |
|
|
When a stop signal appears, resume first and do not continue to the next step.
The training set cannot see future data
This section only deals with “the training set cannot see future data”. Save the current state first and then run the minimal test. Do not change multiple conditions at the same time.
|
|
For Kronos, this boundary must be combined when judging the results: Kronos is a financial K-line modeling project; the forecast output cannot be used as investment advice, and data leakage, re-righting methods, transaction costs and out-of-sample testing will significantly change the backtest results.
If the results are not repeatable, record the version, input, and logs, then narrow the scope of the test.
Rolling backtesting is better than a random split
It is recommended to execute in the following order:
- Check the actual version and source.
- Use a test account, test directory, or test device.
- Perform an undoable action.
- Save the output, exit code, or screenshot.
- Revert the changes and re-verify.
|
|
The completion standard is that “rolling backtesting is better than a random split” to obtain verifiable results, not that the command happens to report no errors.
Add fees and slippage to results
| Check items | Passing criteria | Stop signal |
|---|---|---|
| Add fees and slippage to results | Clear input, output and authority scope | Tool automatically expands operating scope |
| Source | Can return to the official warehouse or original data | Rely on unknown mirrors or second-hand conclusions |
| Security | No secret value in log | Token, cookie or private data present |
| Recovery | Ability to return to pre-execution state | No backup or undo methods |
|
|
When a stop signal appears, resume first and do not continue to the next step.
Compare simple benchmarks instead of just looking at models
This section only deals with “comparing simple benchmarks rather than just looking at models”. Save the current state first and then run the minimal test. Do not change multiple conditions at the same time.
|
|
For Kronos, this boundary must be combined when judging the results: Kronos is a financial K-line modeling project; the forecast output cannot be used as investment advice, and data leakage, re-righting methods, transaction costs and out-of-sample testing will significantly change the backtest results.
If the results are not repeatable, record the version, input, and logs, then narrow the scope of the test.
Save random seeds and data snapshots
It is recommended to execute in the following order:
- Check the actual version and source.
- Use a test account, test directory, or test device.
- Perform an undoable action.
- Save the output, exit code, or screenshot.
- Revert the changes and re-verify.
|
|
The completion standard is to “save the random seed and data snapshot” to obtain verifiable results, rather than the command just not reporting an error.
Reasons why the prediction direction is correct but unable to trade
| Check items | Passing criteria | Stop signal |
|---|---|---|
| Reasons why the prediction direction is correct but cannot be traded | The input, output and authority ranges are clear | The tool automatically expands the scope of operations |
| Source | Can return to the official warehouse or original data | Rely on unknown mirrors or second-hand conclusions |
| Security | No secret value in log | Token, cookie or private data present |
| Recovery | Ability to return to pre-execution state | No backup or undo methods |
|
|
When a stop signal appears, resume first and do not continue to the next step.
How should research conclusions be expressed?
This section only deals with “how research conclusions should be expressed.” Save the current state first and then run the minimal test. Do not change multiple conditions at the same time.
|
|
For Kronos, this boundary must be combined when judging the results: Kronos is a financial K-line modeling project; the forecast output cannot be used as investment advice, and data leakage, re-righting methods, transaction costs and out-of-sample testing will significantly change the backtest results.
If the results are not repeatable, record the version, input, and logs, then narrow the scope of the test.
Training data supplier changes will destroy comparability
It is recommended to execute in the following order:
- Check the actual version and source.
- Use a test account, test directory, or test device.
- Perform an undoable action.
- Save the output, exit code, or screenshot.
- Revert the changes and re-verify.
|
|
The completion standard is that “changes in training data suppliers will destroy comparability” and obtain verifiable results, rather than that the command happens to report no errors.
Prediction intervals and trading holding periods cannot be mixed.
| Check items | Passing criteria | Stop signal |
|---|---|---|
| Prediction intervals and trading holding periods cannot be mixed | Input, output and authority scopes are clear | Tools automatically expand the scope of operations |
| Source | Can return to the official warehouse or original data | Rely on unknown mirrors or second-hand conclusions |
| Security | No secret value in log | Token, cookie or private data present |
| Recovery | Ability to return to pre-execution state | No backup or undo methods |
|
|
When a stop signal appears, resume first and do not continue to the next step.
The backtest report must retain the failure interval
This section only deals with “the backtest report must retain the failure interval”. Save the current state first and then run the minimal test. Do not change multiple conditions at the same time.
|
|
For Kronos, this boundary must be combined when judging the results: Kronos is a financial K-line modeling project; the forecast output cannot be used as investment advice, and data leakage, re-righting methods, transaction costs and out-of-sample testing will significantly change the backtest results.
If the results are not repeatable, record the version, input, and logs, then narrow the scope of the test.
Final Check with Kronos
- The versions and sources used are documented.
- Test input and formal data have been isolated.
- Commands, configurations and actual output can correspond.
- The failed path was actively tested at least once.
- Keys, accounts and private data do not enter Git or logs.
- There are clear actions for upgrading, stopping and resuming.
If you still can’t explain how the results occurred, keep Kronos in a test environment without extending permissions or replacing existing stable processes.