docs: premium UI improvements — navbar links, hover effects, inline tips, accordion troubleshooting (#646)

- Move Discord, GitHub, PyPI, and Follow on X links from sidebar anchors to top-right navbar
- Lock dark mode as default via appearance.strict and hide theme toggle
- Add custom.css with hover highlighting for tables, code blocks, cards, callouts, and inline code
- Move all Tips and Common Pitfalls sections inline next to their relevant content across all 25 reference docs
- Polish context.md: remove duplicates, condense callouts, upgrade Cookbooks to CardGroup
- Convert Troubleshooting and Performance Optimization sections in installation.md, cli-setup.md, explorer-setup.md, learning-more.md, and faq.md from plain headers to AccordionGroup
- Change navigation-hint Tip callouts to Info in concepts.md, faq.md, glossary.md, and modules.md
This commit is contained in:
Mohd Kaif
2026-06-17 18:59:25 +05:30
committed by GitHub
parent a326c7d3bd
commit e04dc12e6e
30 changed files with 723 additions and 593 deletions
+12 -14
View File
@@ -179,6 +179,10 @@ splitter = TextSplitter(
| `relation_method` | `str` | `"ml"` | Relation extraction method for `relation_aware`: `"ml"` \| `"llm"` \| `"huggingface"` |
| `tokenizer` | `str` | `"gpt-4"` | tiktoken model name for `token` method: unrecognised names fall back to `cl100k_base` |
<Warning>
**`chunk_overlap` too small.** Without overlap, a fact that spans a chunk boundary is invisible in both chunks. A 1020% overlap relative to `chunk_size` is a safe minimum: for `chunk_size=1000`, set `chunk_overlap=100` to `200`.
</Warning>
## Splitting Method Details
<Tabs>
@@ -217,6 +221,10 @@ splitter = TextSplitter(
- Produces variable-length chunks: some topics are short, others long
- Falls back to sentence splitting if `sentence-transformers` is not installed
- Slower than `recursive` due to embedding computation; cache embeddings for repeated splits
<Tip>
**Semantic splitting needs enough sentences.** `semantic_transformer` needs several sentences to detect topic shifts. On documents shorter than ~300 words it behaves like `sentence` splitting: use `recursive` instead.
</Tip>
</Tab>
<Tab title="Entity-Aware">
Runs NER internally, then adjusts chunk boundaries so no entity mention is split across two chunks:
@@ -346,6 +354,10 @@ The `token` method accepts a `tokenizer=` kwarg that is passed to `tiktoken.enco
If `tiktoken` is not installed, the `token` method falls back to splitting by whitespace-separated words.
<Warning>
**Wrong tokenizer.** The `token` method passes the `tokenizer=` value to `tiktoken.encoding_for_model()`. If the model name is not recognised by tiktoken it silently falls back to `cl100k_base`. Pass a valid tiktoken model name (e.g. `"gpt-4"`, `"gpt-3.5-turbo"`) to get deterministic behaviour.
</Warning>
## Pipeline Integration
`TextSplitter` can be used standalone or composed manually with other Semantica modules. The example below shows a sequential pattern: parse a file, split the text, then extract entities from each chunk:
@@ -373,20 +385,6 @@ for chunk in chunks:
For the full pipeline orchestration API, see the [Pipeline reference](pipeline).
## Tips and Common Pitfalls
<Warning>
**`chunk_overlap` too small.** Without overlap, a fact that spans a chunk boundary is invisible in both chunks. A 1020% overlap relative to `chunk_size` is a safe minimum: for `chunk_size=1000`, set `chunk_overlap=100` to `200`.
</Warning>
<Warning>
**Wrong tokenizer.** The `token` method passes the `tokenizer=` value to `tiktoken.encoding_for_model()`. If the model name is not recognised by tiktoken it silently falls back to `cl100k_base`. Pass a valid tiktoken model name (e.g. `"gpt-4"`, `"gpt-3.5-turbo"`) to get deterministic behaviour.
</Warning>
<Tip>
**Semantic splitting needs enough sentences.** `semantic_transformer` needs several sentences to detect topic shifts. On documents shorter than ~300 words it behaves like `sentence` splitting: use `recursive` instead.
</Tip>
<CardGroup cols={2}>
<Card title="Parse" icon="file-lines" href="parse">
Parse documents before chunking: produces sections and metadata.