DeepTutor knowledge base and RAG configuration: document import, Embedding, memory auditing and backup

DeepTutor knowledge base and RAG configuration: document import, Embedding, memory auditing and backup, covering configuration, verification, permission boundaries, failback and long-term maintenance.

This tutorial by DeepTutor only deals with the specific task in the title. The quality of the knowledge base depends on document cleaning, Embedding consistency and reference review; DeepTutor’s settings, logs, memory and knowledge base are placed in the data directory of the workspace by default, and the data life cycle should be designed clearly first.

All the following operations are first placed in the test repository, test account or service that listens only on the loopback interface. The domain name, username, path and key in the command are placeholders and need to be replaced before execution.

First fix the position of DeepTutor workspace

This section solves the problem of “fixing the position of the DeepTutor workspace first”. First record the current state, then perform the minimum action, and finally confirm the result with independent evidence.

For DeepTutor, the basis for judgment is: the quality of the knowledge base depends on document cleaning, Embedding consistency and reference review; DeepTutor’s settings, logs, memory and knowledge base are placed in the data directory of the workspace by default, and the data life cycle should be clearly designed first. Don’t accidentally open more permissions at this stage.

1
$env:DEEPTUTOR_HOME='D:\deeptutor-data'

Preserve command output and timestamp after execution. If the output depends on temporary variables in the current terminal, open a new terminal and check again.

Model and Embedding are two sets of configurations

Process in the following order:

  1. Read the actual version and current configuration.
  2. Change only one setting relevant to this section.
  3. Run a read-only or revocable request.
  4. Check logs, exit codes, and final files.
  5. Revert the previous modification if it fails.
1
deeptutor init

The completion standard here is not the interface appearance, but “model and Embedding are two sets of configurations” with repeatable results.

Clean PDF and Markdown before importing

What to check Acceptable performance Signals that you need to stop
Clean PDF and Markdown before import Clear input and output scope Automatically expand to other projects or accounts
Permissions Get only the permissions you need to complete the task Require administrator rights or full key
Log Failed to locate and has been desensitized Token, Cookie or private text appears
Rollback Can restore the previous state Modifications are irreversible and there is no backup
1
Get-ChildItem -Recurse .\documents

Once the stop signal in the table appears, undo the changes in this section first and do not continue with subsequent automation.

Split the knowledge base by topic instead of mixing them all

Prepare a success sample and a failure sample around “splitting the knowledge base by topic instead of mixing it all in”. Successful samples verify the normal path, and failed samples verify whether the restrictions really take effect.

1
deeptutor start

It is recommended to record the following four items:

  • Pre-execution version or Git commit.
  • Actual input, no secret value recorded.
  • Observable output, status code or diff.
  • Recovery actions and review results after recovery.

If the cause of the failure is still unclear, modify only one variable at a time; do not change the port, runtime, provider, and proxy at the same time.

Do three types of retrieval tests after the first indexing

This section solves the problem of “doing three types of retrieval tests after the first index”. First record the current state, then perform the minimum action, and finally confirm the result with independent evidence.

For DeepTutor, the basis for judgment is: the quality of the knowledge base depends on document cleaning, Embedding consistency and reference review; DeepTutor’s settings, logs, memory and knowledge base are placed in the data directory of the workspace by default, and the data life cycle should be clearly designed first. Don’t accidentally open more permissions at this stage.

1
curl.exe http://127.0.0.1:3782

Preserve command output and timestamp after execution. If the output depends on temporary variables in the current terminal, open a new terminal and check again.

Answers must show evidence that can be reviewed

Process in the following order:

  1. Read the actual version and current configuration.
  2. Change only one setting relevant to this section.
  3. Run a read-only or revocable request.
  4. Check logs, exit codes, and final files.
  5. Revert the previous modification if it fails.
1
deeptutor start

The completion standard here is not that the interface appears, but that “the answer must show evidence that can be reviewed” and have repeatable results.

What do L1, L2, and L3 memories check respectively?

What to check Acceptable performance Signals that you need to stop
L1, L2, and L3 remember what to check respectively The input and output ranges are clear Automatically expand to other projects or accounts
Permissions Get only the permissions you need to complete the task Require administrator rights or full key
Log Failed to locate and has been desensitized Token, Cookie or private text appears
Rollback Can restore the previous state Modifications are irreversible and there is no backup
1
Get-ChildItem -Recurse $env:DEEPTUTOR_HOME

Once the stop signal in the table appears, undo the changes in this section first and do not continue with subsequent automation.

Correct incorrect memory instead of appending

Prepare a success sample and a failure sample around “correcting false memories instead of continuing to append”. Successful samples verify the normal path, and failed samples verify whether the restrictions really take effect.

1
Get-ChildItem -Recurse $env:DEEPTUTOR_HOME | Sort-Object LastWriteTime -Descending | Select-Object -First 20

It is recommended to record the following four items:

  • Pre-execution version or Git commit.
  • Actual input, no secret value recorded.
  • Observable output, status code or diff.
  • Recovery actions and review results after recovery.

If the cause of the failure is still unclear, modify only one variable at a time; do not change the port, runtime, provider, and proxy at the same time.

What data is actually saved in the Docker volume?

This section addresses “what data is actually held in a Docker volume”. First record the current state, then perform the minimum action, and finally confirm the result with independent evidence.

For DeepTutor, the basis for judgment is: the quality of the knowledge base depends on document cleaning, Embedding consistency and reference review; DeepTutor’s settings, logs, memory and knowledge base are placed in the data directory of the workspace by default, and the data life cycle should be clearly designed first. Don’t accidentally open more permissions at this stage.

1
docker volume inspect deeptutor-data

Preserve command output and timestamp after execution. If the output depends on temporary variables in the current terminal, open a new terminal and check again.

Why does it need to be rebuilt when replacing the Embedding model?

Process in the following order:

  1. Read the actual version and current configuration.
  2. Change only one setting relevant to this section.
  3. Run a read-only or revocable request.
  4. Check logs, exit codes, and final files.
  5. Revert the previous modification if it fails.
1
docker stop deeptutor

The completion standard here is not the appearance of the interface, but the reproducible results of “Why do you need to rebuild the Embedding model when replacing it?”

Backup and restore a complete knowledge base

What to check Acceptable performance Signals that you need to stop
Back up and restore a complete knowledge base at one time Clear input and output scope Automatically expand to other projects or accounts
Permissions Get only the permissions you need to complete the task Require administrator rights or full key
Log Failed to locate and has been desensitized Token, Cookie or private text appears
Rollback Can restore the previous state Modifications are irreversible and there is no backup
1
docker run --rm -v deeptutor-data:/data -v ${PWD}:/backup alpine tar czf /backup/deeptutor-data.tgz -C /data .

Once the stop signal in the table appears, undo the changes in this section first and do not continue with subsequent automation.

Sampling evaluation method for long-term maintenance

Prepare a success sample and a failure sample around the “sampling evaluation method for long-term maintenance”. Successful samples verify the normal path, and failed samples verify whether the restrictions really take effect.

1
deeptutor start

It is recommended to record the following four items:

  • Pre-execution version or Git commit.
  • Actual input, no secret value recorded.
  • Observable output, status code or diff.
  • Recovery actions and review results after recovery.

If the cause of the failure is still unclear, modify only one variable at a time; do not change the port, runtime, provider, and proxy at the same time.

How scan OCR errors affect retrieval

This section addresses “How Scanned OCR Errors Affect Retrieval.” First record the current state, then perform the minimum action, and finally confirm the result with independent evidence.

For DeepTutor, the basis for judgment is: the quality of the knowledge base depends on document cleaning, Embedding consistency and reference review; DeepTutor’s settings, logs, memory and knowledge base are placed in the data directory of the workspace by default, and the data life cycle should be clearly designed first. Don’t accidentally open more permissions at this stage.

1
Get-ChildItem -Recurse .\documents

Preserve command output and timestamp after execution. If the output depends on temporary variables in the current terminal, open a new terminal and check again.

Whether Chinese and English materials are shared Embedding

Process in the following order:

  1. Read the actual version and current configuration.
  2. Change only one setting relevant to this section.
  3. Run a read-only or revocable request.
  4. Check logs, exit codes, and final files.
  5. Revert the previous modification if it fails.
1
deeptutor init

The completion standard here is not the appearance of the interface, but the repeatable results of “whether Chinese and English materials share Embedding”.

Index residue after deleting documents

What to check Acceptable performance Signals that you need to stop
Index residue after deleting documents Clear input and output scope Automatically expand to other projects or accounts
Permissions Get only the permissions you need to complete the task Require administrator rights or full key
Log Failed to locate and has been desensitized Token, Cookie or private text appears
Rollback Can restore the previous state Modifications are irreversible and there is no backup
1
Get-ChildItem -Recurse $env:DEEPTUTOR_HOME

Once the stop signal in the table appears, undo the changes in this section first and do not continue with subsequent automation.

Restore to a new workspace

Prepare a success sample and a failure sample around “restoring to a completely new workspace”. Successful samples verify the normal path, and failed samples verify whether the restrictions really take effect.

1
2
$env:DEEPTUTOR_HOME='D:\deeptutor-restore'
deeptutor start

It is recommended to record the following four items:

  • Pre-execution version or Git commit.
  • Actual input, no secret value recorded.
  • Observable output, status code or diff.
  • Recovery actions and review results after recovery.

If the cause of the failure is still unclear, modify only one variable at a time; do not change the port, runtime, provider, and proxy at the same time.

DeepTutor FAQ

Is it possible to skip the test environment and use DeepTutor directly in the official project?

Not recommended. Complete at least one minimal success request, one intentional failure, and one recovery drill first.

The DeepTutor command can be run but the result is incorrect, where should I check first?

First check the input range, actual effective configuration and upstream response, and then check the model summary. A normal process does not mean that the business results are correct.

How to prevent DeepTutor’s keys or tokens from entering Git?

Use system environment variables, secret management, or configuration files outside the project, and search the diff before committing. Keys must be rotated after a breach is discovered.

What is most likely to be missed when upgrading DeepTutor?

It is most easy to miss configuration format, default listening address, permission scope and cache compatibility. Save the version and verification samples before upgrading.

DeepTutor acceptance issues

Upon completion you should be able to answer the following questions:

  • Which exact version are you using?
  • Which directories, ports, accounts and external services can be accessed?
  • How to return the successful result to the original data or Git diff?
  • When the upstream fails, will it report an error, retry, or switch?
  • Is it possible that the key appears in logs or history?
  • How to return to the pre-modification state within ten minutes?

If any of these cannot be answered, DeepTutor is still in trial status and should not be extended permissions or connected to production automation.