Personal Website π
I’ve been trying to figure out the issues with Search Engine Indexing. I noticed this problem a while ago. The problem is that Google is gradually removing my blog pages from its database. You can see it on the graph: the number of indexed pages is going down.
- I completely removed the Notify URL API from my code since there’s no benefit to it.
- I will be re-doing manual indexing with a crawler.
Goal: reach an indexing level of 99%…100%.
It seems that a way to automate Google indexing via API actually doesn’t exist. You either enter the URL manually in Google Search Console, or you wait until Google decides your site is popular enough and deigns to index it.
LLM KV-cache quant π
I experimented with KV quants. By default, we have K=F16, V=F16. Dropping down to Q8 provides a noticeable speed boost, especially when there’s a lack of VRAM. However, after testing, I decided itβs not worth it because the loss in precision is too great. I read a comparison like this: when you lower the KV, it’s as if the model puts on blurry glasses, looks through fog, and can’t make out the words in the input data. It’s better to leave KV at F16 and endure the lower speed.
GPU Retrospective, RX 7900 XT π₯οΈ
Last year, I bought a new video card for LLMs and have been using it ever since. The old card had 16 GB of VRAM on board, and the new one has 20 GB. At the time of purchase, I doubted if it was worth it. Now, after 8 months, I can draw these conclusions:
- π LLMs up to 27B fly
- π° LLMs around 31B work, but a progressively filling context lags heavily
Was it worth it? Yes, but actually no π₯΄
- π’ Yes, because I actively used the video card, so I can’t say it was “gathering dust” π§Ή
- π‘ No, because the upgrade from 16 to 20 GB isn’t that significant. LLMs managed to run okay-ish on the old card too. There were more stutters before, but the difference wasn’t fundamental…
In the end, the main takeaway is this: it would have been more logical to pay extra and go straight for the top video card in the RX7000 series with 24 GB of memory. Or just continue sitting comfortably on 16 GB. There was very little point to the “intermediate” 20 GB solution between 16 GB and 24 GB π§
