Skip to content

Instantly share code, notes, and snippets.

@yukiarimo
Created March 14, 2024 20:52
Show Gist options
  • Save yukiarimo/8be25767362f580434aec0fc39504d3d to your computer and use it in GitHub Desktop.
Save yukiarimo/8be25767362f580434aec0fc39504d3d to your computer and use it in GitHub Desktop.
Web novel Downloader
function extractAndDownloadAllChapters() {
// Find all containers that hold chapter text
const chapterContainers = document.querySelectorAll('.cha-words');
// Initialize an array to hold all chapter texts
let allChaptersText = [];
// Iterate over each chapter container
chapterContainers.forEach(container => {
// Get all paragraph elements within the container
const paragraphs = container.querySelectorAll('p');
// Extract the text from each paragraph and join them with a newline character
const chapterText = Array.from(paragraphs).map(p => p.textContent.trim()).join('\n');
// Add the chapter text to the array
allChaptersText.push(chapterText);
});
// Join all chapters with two newline characters to separate them
const allText = allChaptersText.join('\n\n');
// Create a Blob with the combined text content
const blob = new Blob([allText], {
type: 'text/plain'
});
// Create an anchor element and use it to trigger the download
const anchor = document.createElement('a');
anchor.href = URL.createObjectURL(blob);
anchor.download = 'allChaptersText.txt';
document.body.appendChild(anchor);
anchor.click();
document.body.removeChild(anchor);
}
@yukiarimo
Copy link
Author

About:

This simple script extracts and downloads all chapters from a www.webnovel.com website page.
It finds all chapters and extracts the text from each paragraph within them.

To use it:

  1. Go to the webnovel page with all the chapters you want to download.
  2. Scroll from the first chapter to the last chapter.
  3. When you reach the last chapter, stop scrolling and wait for the page to load all the chapters.
  4. Once all the chapters are loaded, right-click on the page and select "Inspect" to open the developer tools.
  5. Go to the "Console" tab in the developer tools.
  6. Copy and paste the script into the console and press Enter.
  7. The script will extract and download all the chapters as a single text file.
  8. Done!

@VividVidun
Copy link

VividVidun commented Jul 30, 2024

when I pasted it in the console as you said, it didn't do anything.

@VividVidun
Copy link

do I have to do something other than that? (I'm a ROOKIE)

@yukiarimo
Copy link
Author

It would be best to open the novel, as you would open it for reading (to see the actual text of the novel), and scroll to the end to load fully. And only then, run this script!

@VividVidun
Copy link

Kay, I'll try.

@VividVidun
Copy link

It would be best to open the novel, as you would open it for reading (to see the actual text of the novel), and scroll to the end to load fully. And only then, run this script!

When it was loaded, I pasted the script and pressed enter, but it said it was undefined.
Screenshot (67)

@yukiarimo
Copy link
Author

LOL. You just defined a function using function itsName() {}. So, after that you need to call it by it's name in order to run it. In this case, it's extractAndDownloadAllChapters(); without the function keyword!

@jokesmcgee
Copy link

I think I'm doing something wrong; I opened the novel, scrolled all the way to the end, and then copied and pasted the code exactly as is into the console and hit enter.
Screenshot 2024-08-31 132105
Is there something really obvious I'm overlooking?

@yukiarimo
Copy link
Author

Please just run extractAndDownloadAllChapters(); after you pasted the full function init code!

@aarongithu
Copy link

This seems really close to what I've been looking for. Thanks so much!

Is there a way to get it to save chapter numbers and titles, too? As it is, the text is just all one continuous block.

@yukiarimo
Copy link
Author

Didn’t it save them if it was written in the novel?

@aarongithu
Copy link

I'm not sure what you mean, but I started (as an example) here:

https://www.webnovel.com/book/i-can-copy-curses_29252483808354105/

...then I paged down as far as it would let me. Then I pasted the code in Console and used extractAndDownloadAllChapters();

It generated allChaptersText.txt , but none of the chapter titles were there; it was just a single uninterrupted flow of text.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment