You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have succeeded in running your basic example of cpp executor. Currently, it only generated token_ids. Could you provide more instructions on how to turn these token ids into human-readable words?
root@ad3499615250:/workspace/TensorRT-LLM_0_16_0/examples/cpp/executor/build# ./executorExampleBasic ../../../llama/tinyllama-engine/
[TensorRT-LLM][INFO] Engine version 0.16.0 found in the config file, assuming engine(s) built by new builder API.
[TensorRT-LLM][INFO] Initializing MPI with thread mode 3
[TensorRT-LLM][INFO] Initialized MPI
[TensorRT-LLM][INFO] Refreshed the MPI local session
...
[TensorRT-LLM][INFO] [MemUsageChange] Allocated 1.52 GiB for max tokens in paged KV cache (72256).
[TensorRT-LLM][INFO] Output tokens: (1, 2, 3, 4, 29889, 13, 1576, 29871, 29906)
[TensorRT-LLM][INFO] Refreshed the MPI local session
The text was updated successfully, but these errors were encountered:
The text was updated successfully, but these errors were encountered: