Tutorial

I believe that LLMs have the potential to be incredibly useful, however the lack of privacy in online services is a major concern to me. Anything put into any service online will be put back into training in an attempt to make the service as good as possible. This can, and has, led to instances of information that should be private leaking. One such example of this is AI coding assistants outputing API keys. An easy way around privacy concerns is to run the LLM locally. If you haven’t taken a look into doing this it might seem a bit challenging, but a lot of work has been put into making this fairly easy. We’ll use docker compose and existing tools to set up a locally running LLM that is even able to access web results to inform it’s answers.

The fstab file contains the drives that will be mounted at boot time. It contains 6 columns of information that can be divided with tabs or spaces. Column 1: Device: You can use the UUID (Recommended) or device path (UUID=UUID or /dev/device_name) Column 2: Mount Point: Directory you want the drive to be mounted in. This can be a path to anywhere on the system. Column 3: File System: Put the partitions file system, auto is valid if unknown Column 4: Options: Allows to change the behavior of the mounted drive. defaults is a valid option Column 5: Backup Operation (Dump): An outdated backup mechanism and should not be used. Use 0 in this column. Column 6: File System Check Order: The root file system should be set to 1, enter 0 for it not to be checked, enter 2 for all other file systems. Generally use 0. Steps for adding a device: