Integrate your changes into the main KoboldCPP project? #5
Comments
|
From my perspective I've tried broaching the topic several times in several different ways. I also put in a lot of hours into the initial stages of what they currently have for the "docs search" - i.e. testing the different algorithms - before they ultimately landed on a derivation of "minisearch"...stuff like that, but I have not received reciprocal behavior, either someone wanting to contribute to this repository or even give feedback, let alone have any of the functionalities formally incorporated directly into kobold.cpp itself. Every single pull request that I've spent hours on has been rejected in its entirety. AI-driven code is shunned, then the stodginess regarding certain things, I could go on but you get the point. The end result is that it's not worth my time to try to integrate anything with kobold.cpp at this time. I want to be clear that I have zero ill-will for anyone at the project. I firmly believe that anyone who runs a repository is essentially a king within their own fiefdom unless they give others editing capabilities, which kobold.cpp hasn't. Moreover, it's my understanding that kobold.cpp was started as a hobby project, kind of like me and my repositories. Nobody is getting paid here, it's not our job, so at the end of the day there's no ill-will. With that being said, I am entitled to my opinions and setting my comfort level about things and what I choose to devote my time to...and Kobold is not that at the moment until there's a little more open-mindedness there and a willingness to reciprocate time spent some more. |
|
Can you link what other commits/discussions have you engaged in over there, beyond the one I found above? From what I can tell in the thread I found there, the main gripe that was brought up was you renaming and structuring the logic in a less straightforward manner. Again, correct me if there is more to this in other threads, but if the code you provide would pass the desired coding standards then it would effectively be just as good as hand-made code, no? I.e. they ultimately feared the addition of unsanitized code from what they viewed as a beginner coder, not the idea itself. Also, if a compromise really cannot be found (although at this point I fail to see why) maybe it is worth just asking them to do it from scratch instead of necessarily contributing to it directly. The main idea would be to show the advantages of this data structure (and I am hoping that indeed it is objectively better and that this isn't, actually, the red-herring reason why it was not considered for addition already!) over what we use currently. Finally correct me if I'm wrong but is this not already present in some form within SillyTavern? |
|
Just a variety of discussions on Discord as well, don't really want to berate them too much because they're decent people. I just can't afford to spend any more time working on stuff unless there's some kind of reciprocity or appreciation or collaboration. There's no real collaboration. The person who approves stuff doesn't contribute at all to other peoples' enhancements or would-be enhancements. There have been multiple times when I've worked on something (not just formal pull requests even) for multiple hours to no constructive feedback or collaboration whatsoever; quite the opposite. That's their right and I don't regret doing stuff like that. Every time I try to contribute I write that time off with no expectations. However, at a certain point, if people want me to spend my valuable time working on their projects I have a few reciprocal demands of my own. ;-) |
|
I understand your point, and is indeed valid of that is the case. I just find it a bit odd as I can't say I've seen such treatment before on other user contributions that I've lurked or suggested. What about the other points I asked above? |
|
What other points? |
|
Oh yeah, there were multiple other commits, but they went to the "experimental" branch I think. That wasn't as important to me as trying to work on other things via the Discord server though... I don't feel that identical code that was generated by AI would be treated equally as something written 100% by a human by Kobold... As far as features, I could request them, but I don't use Kobold really anymore except in my program as the "Ask Jeeves" functionality. See here. https://github.com/BBC-Esq/VectorDB-Plugin-for-LM-Studio Hope that answers all the questions! Again, they've never been rude to me, I'm not personally offended, I don't have ill will towards them. I still enjoy hanging out with them. it's more a practical consideration thing. I don't want to contribute any more of my time is all unless something changes. |
|
For example, it's more valuable to spend my time doing stuff like this...Currently processing over 130,000 court cases dating back to the mid 19th century into a vector database, stuff like that. :-P
Only 5.5 more hours to go! (and I'm on an RTX 4090 BTW). |
|
Yeah I played a bit with rag back when nvidia released their proof of concept frontend for ingesting and querrying documents like this (which obviously was a huge turd of an implementation, limited to hell and back). Thanks for the clarifications above as well, and I'm sorry you guys didn't manage to work something out. The argument about ai code not being "treated equally as something written 100% by a human" I still think should not hold water for that reason alone - good code is good code, bad code is bad code, no matter the source - but it's a moot point at this.. point, and I understand you wanting to move on. I still hope one day kobold will get this functionality as well, it's already a very useful swiss-army knife of features wrapped in a very accessible and sleek package, something that 99% of other projects lack terribly. |
|
YOU HAVE TO BE KIDDING ME https://github.com/LostRuins/koboldcpp/releases/tag/v1.81
Not really to the level of VectorDB but here's hope it will eventually be considered! I know you made your point very clear about not wanting to bother, but if you're not gonna use this opportunity to nag him about adding (your) VecteorDB support then I will XD |
|
Yes, I'm aware, and I put a lot of work into helping them understand how RAG works, testing various backends, etc. that ultimately led to this implementation. You might characterize my role as a "consultant." lol. However, the primary person responsible for it named @JaXXX was basically who's responsible for it's integration. Anyways, it's good they finally have something "RAG-like." It'll never be as accurate as true RAG for "semantic" searches, but it keeps with the lightweight nature of Kobold. Integrating true RAG would likely require modifying koboldcpp itself, not merely kobold-lite though due to dependency size constraints. |
|
The release does credit "esolithe," which is good, that's the same person as @JaXXX on the discord server so. |
|
I'm glad we at least got -something- lol. Now only if the new web search was better documented about how to trigger... |
Given the quite... manual installation steps required to get your modifications and kobold running, would it not be more feasible to suggest LostRuins to integrate your changes into the main koboldcpp project?
I found that you already mentioned your plugin here on a pull request but it did not seem to go past that, and that's been half a year ago. Any updates on this? Maybe you could ask him to help you a bit? Integrating this feature and thus getting a more robust long-term memory sounds like a very useful thing to have!
The text was updated successfully, but these errors were encountered: