7c9792a6e0
Support unicode characters in model path ( #3681 )
...
* parse wide argv characters on windows
* cleanup
* move cleanup to end of `main`
2024-04-16 17:00:12 -04:00
0a0e9f3e0f
Apply 01-cache.diff
2024-04-01 16:48:18 -07:00
58d95cc9bd
Switch back to subprocessing for llama.cpp
...
This should resolve a number of memory leak and stability defects by allowing
us to isolate llama.cpp in a separate process and shutdown when idle, and
gracefully restart if it has problems. This also serves as a first step to be
able to run multiple copies to support multiple models concurrently.
2024-04-01 16:48:18 -07:00
f5ca7f8c8e
add license in file header for vendored llama.cpp code ( #3351 )
2024-03-26 16:23:23 -04:00
43799532c1
Bump llama.cpp to b2474
...
The release just before ggml-cuda.cu refactoring
2024-03-23 09:54:56 +01:00
e95ffc7448
llama: remove server static assets ( #3174 )
2024-03-15 19:24:12 -07:00
85129d3a32
Adapt our build for imported server.cpp
2024-03-12 14:57:15 -07:00
9ac6440da3
Import server.cpp as of b2356
2024-03-12 13:58:06 -07:00
53c107e20e
chore: fix typo ( #3073 )
...
Signed-off-by: racerole <jiangyifeng@outlook.com >
2024-03-12 14:09:22 -04:00
b80661e8c7
relay load model errors to the client ( #3065 )
2024-03-11 16:48:27 -04:00
369eda65f5
update llama.cpp submodule to ceca1ae ( #3064 )
2024-03-11 12:57:48 -07:00
1ffb1e2874
update llama.cpp submodule to 77d1ac7 ( #3030 )
2024-03-09 15:55:34 -08:00
0e4669b04f
update llama.cpp submodule to 6cdabe6 ( #2999 )
2024-03-08 00:26:20 -08:00
21347e1ed6
update llama.cpp submodule to c29af7e ( #2868 )
2024-03-01 15:26:04 -08:00
4613a080e7
update llama.cpp submodule to 66c1968f7 ( #2618 )
2024-02-20 17:42:31 -05:00
01ff2e14db
[nit] Remove unused msg local var. ( #2511 )
2024-02-20 14:02:34 -05:00
f7231ad9ad
set shutting_down to false once shutdown is complete ( #2484 )
2024-02-13 17:48:41 -08:00
6680761596
Shutdown faster
...
Make sure that when a shutdown signal comes, we shutdown quickly instead
of waiting for a potentially long exchange to wrap up.
2024-02-08 22:22:50 -08:00
72b12c3be7
Bump llama.cpp to b1999
...
This requires an upstream change to support graceful termination,
carried as a patch.
2024-01-30 16:52:12 -08:00
730dcfcc7a
Refine debug logging for llm
...
This wires up logging in llama.cpp to always go to stderr, and also
turns up logging if OLLAMA_DEBUG is set.
2024-01-22 12:26:49 -08:00
ec3764538d
Probe GPUs before backend init
...
Detect potential error scenarios so we can fallback to CPU mode without
hitting asserts.
2024-01-21 15:59:38 -08:00
1b249748ab
Add multiple CPU variants for Intel Mac
...
This also refines the build process for the ext_server build.
2024-01-17 15:08:54 -08:00
557110d0ba
Disable mmap with lora layers ( #1985 )
2024-01-13 23:36:31 -05:00
8da7bef05f
Support multiple variants for a given llm lib type
...
In some cases we may want multiple variants for a given GPU type or CPU.
This adds logic to have an optional Variant which we can use to select
an optimal library, but also allows us to try multiple variants in case
some fail to load.
This can be useful for scenarios such as ROCm v5 vs v6 incompatibility
or potentially CPU features.
2024-01-10 17:27:51 -08:00
2c6e8f5248
Update submodule to 6efb8eb30e7025b168f3fda3ff83b9b386428ad6 ( #1885 )
...
* update submodule to `6efb8eb30e7025b168f3fda3ff83b9b386428ad6`
* unblock condition variable in `update_slots` when closing server
2024-01-10 16:48:38 -05:00
dbdd50b283
add -DCMAKE_SYSTEM_NAME=Darwin cmake flag ( #1832 )
2024-01-07 00:46:17 -05:00
77d96da94b
Code shuffle to clean up the llm dir
2024-01-04 12:12:05 -08:00
