Cyoc down

[godleydemon]

So as I'm sure alot of people have noticed, CYOC has gone down now for several days in a row. Looks like there database is down or something. Ironically, I've been afraid of sites disappearing lately and was in the process of making my own kind of backup database of the chapters. I've got about 3500+ entries. I don't scan every day and I respect there wait rules which is 10 seconds, so I don't end up ddosing there site lol

I wonder if anyone knows what in the world is going on? Also, don't suppose anyone has there own backup database of cyoc? and if people are curious I'm more than happy to share my own spaghetti code in python on how I did it

[lmorgado]

It's strange on my end too. I just finished and overly long and creative chapter and the moment to submitted, the site went down, and this was on a Tuesday or Wednesday when it seemed okay at first. I was lucky to copy and paste the story on Notes before hand, otherwise I would've lost so much writing. Hopefully, they get it back up, but with it taking this long, I fear we might see the end of CYOC as a whole, if not now, than soon.

[godleydemon]

getting a bit scary now, I'm starting to wonder if CYOC is gone for good. Which would be super unlucky since I was just in the middle of trying to backup all of the posts might poke around tonight and try and track down someone who might know whats going on. Unless anyone heres got a clue.

[godleydemon]

so, with cyoc being down now for about a week. I've decided to try and figure out a work around/pull my hair out. I've found the wayback machine seems to have archived a bunch of chapters. I mean ALOT of chapters. Unfortunately, it's still not everything. Thankfully wayback has an api called "wayback cdx server" that I can interface with to search for the latest date of a url with some wild cards. It's a lot more clunky now, and wayback machine has some really odd rate limiting going on. But it IS grabbing chapters once again. Also unfortunately, the date that I was saving in the database for the chapter is going to be wildly off now XD lol So is the ID I was saving, I can probably fix some of this but I don't think it'll be all that useful now with the CYOC site being down for the foreseable future.

It's going to take me a VERY long time to grab all of this data and sort through some of it. I'm making a seperate python script to give me a gui to easily search it. Maybe I'll release it all once I get it done with, so we at least have something we can have fun sifting around inside of. Heck, I know some people would love to use it for AI driven projects

What I'm saving is the id, url, title, branch, date and chapter contents. I am not saving the "what to do next" chapter select. I might be able to do this by creating a new table in the db or something. Maybe something for a future project, if there is enough demand for it. The original project was really only supposed to be for myself, so I had something easy to search through using synonyms of keywords. Then if I was interested in the chapter, I could click a button to go to that chapter and add a new option if I wanted to. It was never designed to try and duplicate the functionality of the site.

If you have any suggestions or ideas on what should be included in the project I'm somehow starting. Please don't hesitate to throw it out there. More than 80% of this code is spaghetti and strung together, working on hopes, dreams and chewing gum. Also, I know for a fact someone on this forum has a database of cyoc chapters. If anyone knows where it's at on the forums (because I can't freaking find the damn thing) let me know.


So it begins

[BBB]

Good luck! Wouldn't want to lose CYOC, so I'm glad you're trying to preserve the content in some form.

[lmorgado]

Well, giving that it is taking so long for CYOC to fix the matter, the chances of it coming back are getting slimmer and slimmer. Godley, can you find my interactive stories and send them to me here. Think it's under my author name of LuisJM. If not, provide me the Wayback link with the CYOC site and I'll find them myself.

[godleydemon]

Originally Posted by lmorgado

Well, giving that it is taking so long for CYOC to fix the matter, the chances of it coming back are getting slimmer and slimmer. Godley, can you find my interactive stories and send them to me here. Think it's under my author name of LuisJM. If not, provide me the Wayback link with the CYOC site and I'll find them myself.

My script isn't saving the author name, since it was just never designed for it. But I'll do you a solid. I'll look for an efficient way to try and scan pages for the author name and download what I find. Now, some pluses and minuses. wayback machine doesn't have everything. In fact, it started scanning chapters at 100,000 which means that's about as many as I'm going to miss, forever (this isn't 100% confirmed yet, since I've never inspected the array that I'm grabbing from wayback to build my list of downloadable objects). Wayback simply doesn't have the chapters saved before that. Also, searching the web archive manually is a bit tedious when it comes to cyoc since some chapters will have options added over time, yet wayback never updated that chapter. So you lose all of the hyperlinks to new chapters that were posted. Even though those chapters exist in waybacks database, you can't navigate to them easily.

Pluses, you CAN manually navigate the cyoc site through web archive. But it's going to be tedious like I said and you're not going to be able to get to everything. If your patient enough you can probably find alot of what your searching for you can get to the cyoc interactive for 2025 here: https://web.archive.org/web/20250328...tory_index.php

[JBird]

Originally Posted by godleydemon

My script isn't saving the author name, since it was just never designed for it. But I'll do you a solid. I'll look for an efficient way to try and scan pages for the author name and download what I find. Now, some pluses and minuses. wayback machine doesn't have everything. In fact, it started scanning chapters at 100,000 which means that's about as many as I'm going to miss, forever (this isn't 100% confirmed yet, since I've never inspected the array that I'm grabbing from wayback to build my list of downloadable objects). Wayback simply doesn't have the chapters saved before that. Also, searching the web archive manually is a bit tedious when it comes to cyoc since some chapters will have options added over time, yet wayback never updated that chapter. So you lose all of the hyperlinks to new chapters that were posted. Even though those chapters exist in waybacks database, you can't navigate to them easily.

Pluses, you CAN manually navigate the cyoc site through web archive. But it's going to be tedious like I said and you're not going to be able to get to everything. If your patient enough you can probably find alot of what your searching for you can get to the cyoc interactive for 2025 here: https://web.archive.org/web/20250328...tory_index.php

There was one thread of chapters I managed to copy and save to a doc, mainly because it's one of few I contributed to. I'd have to double-check the archive for the authors' names, because I noticed they would sometimes not load in.

[godleydemon]

to give a quick update about this small project. I have temporarily stopped scanning for new pages on the web archive. I'm now running a separate script that goes through and updates the author of what I have. Since I've decided that it would take way to freaking long to wait for the initial scan to finish and then start adding authors.

I'm currently around 170,000+ entries in my db, so even updating all of the authors is going to take a long ass while. Probably the rest of the week by my estimates. Once I get all of the authors for what I currently already have in my db. Then I'll update my viewer to include the ability to search by author. Maybe a checkbox or something.

Now, onto an interesting problem I've run into that in hindsight I probably should have expected. At 170,000+ rows on the sqlite db, because of the way I'm searching, it takes about 4 - 5 minutes to complete a full search. Now for the why this is happening. It's because I'm not using simplified string searches, I'm using regex ontop of synonym phrases. So if someone searches "Breasts shrinking" it also searching for: breasts getting smaller, chest shrinking, bra getting loose, ect. and it tries to match those exact phrases. So I'm performing one giant regex search on a row and it takes a sec. I've already sped this WAY up since version 0.5 when I increased the accuracy of results by adding multi threading and a few other tricks.

I'll be releasing version 1 like this, but I'll eventually include a version 2. Version 2 will have an interesting feature that should speed up searches. The idea actually came from a buddy of mine. When someone does a synonym search, version 2 will apply a new tag to the row it finds the synonym in. It'll also save the id in a separate json array that's basically saying, "I found a synonym of breast shrinking in the contents of this row id" this way, in the future, even if new entries are added to the db. It'll always bring those search results up quickly. It doesn't need to rescan other contents either, since I plan to create a sort of checkpoint in the array. You just search "breasts shrinking" and you should get results within seconds instead of minutes. But that's for version 2. I'll eventually make all of this code public and easily accessible. I'm still adding comments to the code and reorganizing it to make it easier to read. right now it looks more like spaghetti. lol

[BBB]

All that backend stuff may be lost on me. But I’m rooting for you! And probably a lot of lurkers and people not even on this site are, too.