v2.3.10: 搭建readthedocs文档网站并迁移所有文档,新增插件—下载图片后缀过滤器,优化代码。(#152)
0
release.py → .github/release.py
vendored
2
.github/workflows/download.yml
vendored
@ -6,7 +6,7 @@ on:
|
||||
paths:
|
||||
- '.github/workflows/download.yml' # 工作流定义
|
||||
- 'usage/workflow_download.py' # 下载脚本
|
||||
- 'assets/config/option_workflow_download.yml' # 配置文件
|
||||
- 'assets/option/option_workflow_download.yml' # 配置文件
|
||||
|
||||
jobs:
|
||||
crawler:
|
||||
|
2
.github/workflows/release_auto.yml
vendored
@ -23,7 +23,7 @@ jobs:
|
||||
id: tb
|
||||
run: |
|
||||
commit_message=$(git log --format=%B -n 1 ${{ github.sha }})
|
||||
python release.py "$commit_message"
|
||||
python .github/release.py "$commit_message"
|
||||
|
||||
- name: Create Release
|
||||
uses: softprops/action-gh-release@v1
|
||||
|
6
.github/workflows/test_api.yml
vendored
@ -10,19 +10,19 @@ on:
|
||||
- 'src/**/*.py'
|
||||
- 'tests/**/*.py'
|
||||
- '.github/workflows/test_api.yml'
|
||||
- 'assets/config/option_test_api.yml'
|
||||
- 'assets/option/option_test_api.yml'
|
||||
|
||||
jobs:
|
||||
test: # This code is based on https://github.com/gaogaotiantian/viztracer/blob/master/.github/workflows/python-package.yml
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: [ 3.7, 3.8, 3.9, "3.10", "3.11" ]
|
||||
python-version: ['3.7', '3.8', '3.11', '3.12']
|
||||
os: [ ubuntu-latest ]
|
||||
runs-on: ${{ matrix.os }}
|
||||
timeout-minutes: 5
|
||||
env:
|
||||
# 配置文件路径
|
||||
JM_OPTION_PATH_TEST: ./assets/config/option_test_api.yml
|
||||
JM_OPTION_PATH_TEST: ./assets/option/option_test_api.yml
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
|
6
.github/workflows/test_html.yml
vendored
@ -10,19 +10,19 @@ on:
|
||||
- 'src/**/*.py'
|
||||
- 'tests/**/*.py'
|
||||
- '.github/workflows/test_html.yml'
|
||||
- 'assets/config/option_test_html.yml'
|
||||
- 'assets/option/option_test_html.yml'
|
||||
|
||||
jobs:
|
||||
test: # This code is based on https://github.com/gaogaotiantian/viztracer/blob/master/.github/workflows/python-package.yml
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: [ 3.7, 3.8, 3.9, "3.10", "3.11" ]
|
||||
python-version: ['3.7', '3.8', '3.11', '3.12']
|
||||
os: [ ubuntu-latest ]
|
||||
runs-on: ${{ matrix.os }}
|
||||
timeout-minutes: 5
|
||||
env:
|
||||
# 配置文件路径
|
||||
JM_OPTION_PATH_TEST: ./assets/config/option_test_html.yml
|
||||
JM_OPTION_PATH_TEST: ./assets/option/option_test_html.yml
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
|
15
.readthedocs.yaml
Normal file
@ -0,0 +1,15 @@
|
||||
# Required
|
||||
version: 2
|
||||
|
||||
# Set the OS, Python version and other tools you might need
|
||||
build:
|
||||
os: ubuntu-22.04
|
||||
tools:
|
||||
python: "3.12"
|
||||
mkdocs:
|
||||
configuration: assets/docs/mkdocs.yml
|
||||
|
||||
# Optionally declare the Python requirements required to build your docs
|
||||
python:
|
||||
install:
|
||||
- requirements: assets/docs/requirements.txt
|
59
README.md
@ -13,10 +13,10 @@
|
||||
```shell
|
||||
pip install jmcomic -i https://pypi.org/project --upgrade
|
||||
```
|
||||
* 本地安装
|
||||
* 通过源代码安装
|
||||
|
||||
```shell
|
||||
pip install -e ./
|
||||
pip install git+https://github.com/hect0x7/JMComic-Crawler-Python
|
||||
```
|
||||
|
||||
## 快速上手
|
||||
@ -26,11 +26,6 @@
|
||||
```python
|
||||
import jmcomic # 导入此模块,需要先安装.
|
||||
jmcomic.download_album('422866') # 传入要下载的album的id,即可下载整个album到本地.
|
||||
# 上面的这行代码,还有一个可选参数option: JmOption,表示配置项,
|
||||
# 配置项的作用是告诉程序下载时候的一些选择,
|
||||
# 比如,要下载到哪个文件夹,使用怎样的路径组织规则(比如[/作者/本子id/图片] 或者 [/作者/本子名称/图片]).
|
||||
# 如果没有配置,则会使用 JmOption.default(),下载的路径是[当前工作文件夹/本子名称/图片].
|
||||
# 如果你想要配置,请参考文件 assets/config/常用配置介绍.yml
|
||||
```
|
||||
|
||||
* v2.2.9: 新增命令行调用方式,上述的代码可以转为一行命令
|
||||
@ -38,51 +33,45 @@ jmcomic.download_album('422866') # 传入要下载的album的id,即可下载
|
||||
```bash
|
||||
# 下载album_id为422866的本子
|
||||
$ jmcomic 422866
|
||||
# 更多用法请参考文件 usage/usage_cl.py (命令行使用介绍)
|
||||
```
|
||||
|
||||
## 进阶使用
|
||||
|
||||
进阶使用可以参考本repo下usage文件夹内的示例代码文件,下面是各个文件的作用,你可以挑感兴趣的阅读:
|
||||
|
||||
- API上手介绍: `getting_started.py`
|
||||
- 命令行使用介绍: `usage_cl.py`
|
||||
- 配置客户端的实现(网页端、移动端): `usage_configure_client_impl.py`
|
||||
- 使用API实现简单功能: `usage_simple.py`
|
||||
- 演示jmcomic模块的可自定义功能点: `usage_custom.py`
|
||||
- 使用API的Filter过滤功能: `usage_feature_filter.py`
|
||||
- 演示jmcomic模块的Plugin插件体系: `usage_feature_plugin.py`
|
||||
- 演示一个综合使用实例: `usage_advanced.py`
|
||||
- 包括6个功能需求的介绍、实现方案和完整运行日志
|
||||
- 实现方案非常简洁,充分jmcomic的便利性,以及强大的插件扩展机制
|
||||
|
||||
以及一些趣味用法:
|
||||
|
||||
- 测试你的ip可以访问哪些禁漫域名: `pick_domain.py`
|
||||
- 基于GitHub Actions下载本子: `workflow_download.py`
|
||||
|
||||
## 项目特点
|
||||
|
||||
- **绕过Cloudflare的反爬虫**
|
||||
- 支持使用**命令行**下载本子,无需写Python代码,简单易用
|
||||
- 支持使用**GitHub Actions**下载本子,网页上直接输入本子id就能下载([教程:使用GitHub Actions下载禁漫本子](./assets/docs/教程:使用GitHub%20Actions下载禁漫本子.md))
|
||||
- 用法多样:
|
||||
|
||||
- GitHub Actions:网页上直接输入本子id就能下载([教程:使用GitHub Actions下载禁漫本子](./assets/docs/sources/tutorial/1_github_actions.md))
|
||||
- 命令行:无需写Python代码,简单易用([教程:使用命令行下载禁漫本子](./assets/docs/sources/tutorial/2_command_line.md))
|
||||
- Python代码:最直接的使用方式,需要你有一定的python编程基础
|
||||
- 支持**网页端**和**移动端**两种客户端实现,可通过配置切换(**移动端不限ip兼容性好,网页端限制ip地区但效率高**)
|
||||
- 支持**自动重试和域名切换**机制
|
||||
- **多线程下载**(可细化到一图一线程,效率极高)
|
||||
- 跟进了JM最新的图片分割算法(2023-02-08)
|
||||
- **可配置性强**
|
||||
|
||||
- 不配置也能使用,十分方便
|
||||
- 配置可以从**配置文件**生成,支持多种文件格式
|
||||
- 配置可以从配置文件生成,支持多种文件格式
|
||||
- 配置点有:`请求域名` `客户端实现` `是否使用磁盘缓存` `同时下载的章节/图片数量` `图片格式转换` `下载路径规则` `请求元信息(headers,cookies,proxies)`等
|
||||
- **可扩展性强**
|
||||
|
||||
- **支持Plugin插件,可以方便地扩展功能,以及使用别人的插件**
|
||||
- 目前内置支持的插件有:`登录插件` `硬件占用监控插件` `只下载新章插件` `压缩文件插件`
|
||||
- 目前内置支持的插件有:`登录插件` `硬件占用监控插件` `只下载新章插件` `压缩文件插件` `下载特定后缀图片插件`
|
||||
- 支持自定义本子/章节/图片下载前后的回调函数
|
||||
- 支持自定义debug日志
|
||||
- 支持自定义类:`Downloader(负责调度)` `Option(负责配置)` `Client(负责请求)` `实体类`等
|
||||
|
||||
## 进阶使用
|
||||
|
||||
进阶使用请查阅文档:[文档](https://jmcomic.readthedocs.io/en/latest)
|
||||
|
||||
下面列出一些常用的文档链接:
|
||||
|
||||
* [option配置文件语法](./assets/docs/sources/option_file_syntax.md)
|
||||
* [常用类和方法演示(下载本子、获取实体类、搜索本子)](assets/docs/sources/tutorial/3_demo.md)
|
||||
* [命令行使用教程](assets/docs/sources/tutorial/2_command_line.md)
|
||||
* [GitHub Actions使用教程](./assets/docs/sources/tutorial/1_github_actions.md)
|
||||
* [插件机制](assets/docs/sources/tutorial/6_plugin.md)
|
||||
* [下载过滤器机制](assets/docs/sources/tutorial/5_filter.md)
|
||||
|
||||
## 使用小说明
|
||||
|
||||
* Python >= 3.7
|
||||
@ -97,8 +86,8 @@ $ jmcomic 422866
|
||||
* src:存放源代码
|
||||
|
||||
* jmcomic:`jmcomic`模块
|
||||
* tests:测试目录,存放测试代码,使用unittest
|
||||
* usage:用法目录,存放示例/使用代码
|
||||
* tests:测试目录,存放测试代码,使用unittest
|
||||
* usage:用法目录,存放示例/使用代码
|
||||
|
||||
## 感谢以下项目
|
||||
|
||||
|
@ -1,25 +0,0 @@
|
||||
# 插件的配置示例
|
||||
|
||||
plugin:
|
||||
after_init:
|
||||
- plugin: usage_log # 实时打印硬件占用率的插件
|
||||
kwargs:
|
||||
interval: 0.5 # 间隔时间
|
||||
enable_warning: true # 占用过大时发出预警
|
||||
|
||||
- plugin: login # 登录插件
|
||||
kwargs:
|
||||
username: un
|
||||
password: pw
|
||||
|
||||
- plugin: find_update # 只下载新章插件
|
||||
kwargs:
|
||||
145504: 290266 # 下载本子145504的章节290266以后的新章
|
||||
|
||||
after_album:
|
||||
- plugin: zip # 压缩文件插件
|
||||
kwargs:
|
||||
level: photo # 按照章节,一个章节一个压缩文件
|
||||
filename_rule: Ptitle # 压缩文件的命名规则
|
||||
zip_dir: D:/jmcomic/zip/ # 压缩文件存放的文件夹
|
||||
delete_original_file: true # 压缩成功后,删除所有原文件和文件夹
|
22
assets/docs/mkdocs.yml
Normal file
@ -0,0 +1,22 @@
|
||||
site_name: jmcomic
|
||||
theme:
|
||||
name: readthedocs
|
||||
highlightjs: true
|
||||
plugins:
|
||||
- search
|
||||
- mkdocstrings:
|
||||
custom_templates: templates
|
||||
handlers:
|
||||
# See: https://mkdocstrings.github.io/python/usage/
|
||||
python:
|
||||
paths: [ '../../src/' ]
|
||||
options:
|
||||
docstring_style: sphinx
|
||||
|
||||
|
||||
markdown_extensions:
|
||||
- markdown_include.include:
|
||||
base_path: .
|
||||
- admonition
|
||||
|
||||
docs_dir: sources
|
4
assets/docs/requirements.txt
Normal file
@ -0,0 +1,4 @@
|
||||
mkdocs
|
||||
mkdocstrings[python]
|
||||
markdown-include
|
||||
mkdocs-material
|
@ -1,4 +1,4 @@
|
||||
# 大版本更新内容计划
|
||||
# Plan For Update Content
|
||||
|
||||
| 版本范围 | 更新内容 |
|
||||
|:--------:|:--------------------------------------:|
|
4
assets/docs/sources/api/client.md
Normal file
@ -0,0 +1,4 @@
|
||||
# client
|
||||
|
||||
::: jmcomic.jm_client_impl
|
||||
|
3
assets/docs/sources/api/config.md
Normal file
@ -0,0 +1,3 @@
|
||||
# config
|
||||
|
||||
::: jmcomic.jm_config.JmModuleConfig
|
4
assets/docs/sources/api/download.md
Normal file
@ -0,0 +1,4 @@
|
||||
# download
|
||||
|
||||
::: jmcomic.api
|
||||
|
3
assets/docs/sources/api/entity.md
Normal file
@ -0,0 +1,3 @@
|
||||
# entity
|
||||
|
||||
::: jmcomic.jm_entity
|
3
assets/docs/sources/api/option.md
Normal file
@ -0,0 +1,3 @@
|
||||
# option
|
||||
|
||||
::: jmcomic.jm_option.JmOption
|
3
assets/docs/sources/api/plugin.md
Normal file
@ -0,0 +1,3 @@
|
||||
# plugin
|
||||
|
||||
::: jmcomic.jm_plugin
|
Before Width: | Height: | Size: 40 KiB After Width: | Height: | Size: 40 KiB |
Before Width: | Height: | Size: 118 KiB After Width: | Height: | Size: 118 KiB |
Before Width: | Height: | Size: 93 KiB After Width: | Height: | Size: 93 KiB |
Before Width: | Height: | Size: 69 KiB After Width: | Height: | Size: 69 KiB |
Before Width: | Height: | Size: 124 KiB After Width: | Height: | Size: 124 KiB |
102
assets/docs/sources/index.md
Normal file
@ -0,0 +1,102 @@
|
||||
# jmcomic
|
||||
|
||||
Python API for JMComic(禁漫天堂)
|
||||
|
||||
## Features
|
||||
|
||||
- Bypasses Cloudflare anti-bot measures.
|
||||
- Multiple usage ways:
|
||||
|
||||
- GitHub Actions: Requires only a GitHub account. (See
|
||||
tutorial → [Tutorial - Download Album via GitHub Actions](./tutorial/1_github_actions.md))
|
||||
|
||||
- Command line: No need to write Python code, simple and easy to use. (See tutorial → [Tutorial - Download Album via Command Line](./tutorial/2_command_line.md))
|
||||
- Python code: The most flexible and powerful way, requires some basic knowledge of Python programming.
|
||||
|
||||
- Supports two client implementations: web-based and mobile-based. Switchable through configuration (mobile-based has
|
||||
better IP compatibility, web-based has higher efficiency).
|
||||
- Supports automatic request retry and domain switching mechanism.
|
||||
- Multi-threaded downloading (can be fine-tuned to one thread per image, highly efficient).
|
||||
- Highly configurable:
|
||||
|
||||
- Can be used without configuration, very convenient.
|
||||
- Configuration can be generated from a configuration file, supports multiple file formats.
|
||||
- Configuration options
|
||||
include: `request domain`, `client implementation`, `number of chapters/images downloaded simultaneously`, `image format conversion`, `download path rules`, `request metadata (headers, cookies, proxies)`,
|
||||
and more.
|
||||
|
||||
- Highly extensible:
|
||||
|
||||
- Supports Plugin plugins for easy functionality extension and use of other plugins.
|
||||
- Currently built-in
|
||||
plugins: `login plugin`, `hardware usage monitoring plugin`, `only download new chapters plugin`, `zip compression plugin`.
|
||||
- Supports custom callback functions before and after downloading album/chapter/images.
|
||||
- Supports custom debug logging.
|
||||
- Supports custom core
|
||||
classes: `Downloader (responsible for scheduling)`, `Option (responsible for configuration)`, `Client (responsible for requests)`, `entity classes`,
|
||||
and more.
|
||||
|
||||
## Install
|
||||
|
||||
- Install via official pip source (recommended, and also used for updates):
|
||||
|
||||
```
|
||||
pip install jmcomic -i https://pypi.org/project --upgrade
|
||||
```
|
||||
|
||||
- Install via GitHub code:
|
||||
|
||||
```
|
||||
pip install git+https://github.com/hect0x7/JMComic-Crawler-Python
|
||||
```
|
||||
|
||||
## Getting Started
|
||||
|
||||
### 1. Download album:
|
||||
|
||||
- Python code
|
||||
|
||||
```python
|
||||
import jmcomic
|
||||
# Pass the ID of the album you want to download, and it will download all chapters of the album to your local machine.
|
||||
jmcomic.download_album('422866')
|
||||
```
|
||||
|
||||
- Command line
|
||||
|
||||
```
|
||||
jmcomic 422866
|
||||
```
|
||||
|
||||
### 2. Customize download behavior using an option:
|
||||
|
||||
For example, if you want to convert all downloaded images to the .jpg format, you can create a YAML file with the
|
||||
following content (refer to [option file syntax](./option_file_syntax.md)):
|
||||
|
||||
```yml
|
||||
download:
|
||||
image:
|
||||
suffix: .jpg # Don't forget the '.'
|
||||
```
|
||||
|
||||
Then, use one of the following ways:
|
||||
|
||||
* Python code
|
||||
|
||||
```python
|
||||
from jmcomic import download_album, create_option
|
||||
option = create_option('/path/to/your/optionfile')
|
||||
download_album('422866', option)
|
||||
```
|
||||
|
||||
* Command line
|
||||
|
||||
```
|
||||
jmcomic 422866 --option="/path/to/your/optionfile"
|
||||
```
|
||||
|
||||
## Acknowledgement
|
||||
|
||||
### Image Segmentation Algorithm Code + JMComic Mobile API
|
||||
|
||||
[](https://github.com/tonquer/JMComic-qt)
|
@ -1,3 +1,22 @@
|
||||
# Option File Syntax
|
||||
|
||||
## 1. 配置前需知
|
||||
|
||||
* option有`默认值`,当你使用配置文件来创建option时,你配置文件中的值会覆盖`默认值`。
|
||||
|
||||
因此,在配置option时,不需要配置全部的值,只需要配置特定部分即可。
|
||||
|
||||
* 你可以使用下面的代码来得到option的默认值,你可以删除其中的大部分配置项,只保留你要覆盖的配置项
|
||||
|
||||
```python
|
||||
from jmcomic import JmOption
|
||||
JmOption.default().to_file('./option.yml') # 创建默认option,导出为option.yml文件
|
||||
```
|
||||
|
||||
## 2. option常用配置项
|
||||
|
||||
```yml
|
||||
# 配置客户端相关
|
||||
client:
|
||||
# impl: 客户端实现类,不配默认是html,表示网页端
|
||||
impl: html
|
||||
@ -37,17 +56,17 @@ client:
|
||||
|
||||
# 下载配置
|
||||
download:
|
||||
cache: true # 如果要下载的文件在磁盘上已存在,不用再下一遍了吧?
|
||||
cache: true # 如果要下载的文件在磁盘上已存在,不用再下一遍了吧?默认为true
|
||||
image:
|
||||
decode: true # JM的原图是混淆过的,要不要还原?
|
||||
suffix: .jpg # 把图片都转为.jpg格式
|
||||
decode: true # JM的原图是混淆过的,要不要还原?默认为true
|
||||
suffix: .jpg # 把图片都转为.jpg格式,默认为null,表示不转换。
|
||||
threading:
|
||||
# image: 同时下载的图片数,默认是30张图
|
||||
# 数值大,下得快,配置要求高,对禁漫压力大
|
||||
# 数值小,下得慢,配置要求低,对禁漫压力小
|
||||
# PS: 禁漫网页一次最多请求50张图
|
||||
image: 30
|
||||
# photo: 同时下载的章节数,不配置默认是cpu核心数
|
||||
# photo: 同时下载的章节数,不配置默认是cpu的线程数。例如8核16线程的cpu → 16.
|
||||
photo: 16
|
||||
|
||||
|
||||
@ -68,3 +87,44 @@ dir_rule:
|
||||
|
||||
# 默认规则是: 根目录 / 章节标题 / 图片文件
|
||||
rule: Bd_Ptitle
|
||||
```
|
||||
|
||||
|
||||
## 3. option插件配置项
|
||||
```yml
|
||||
# 插件的配置示例
|
||||
plugins:
|
||||
after_init:
|
||||
- plugin: usage_log # 实时打印硬件占用率的插件
|
||||
kwargs:
|
||||
interval: 0.5 # 间隔时间
|
||||
enable_warning: true # 占用过大时发出预警
|
||||
|
||||
- plugin: login # 登录插件
|
||||
kwargs:
|
||||
username: un # 用户名
|
||||
password: pw # 密码
|
||||
|
||||
- plugin: find_update # 只下载新章插件
|
||||
kwargs:
|
||||
145504: 290266 # 下载本子145504的章节290266以后的新章
|
||||
|
||||
- plugin: image_suffix_filter # 图片后缀过滤器插件,可以控制只下载哪些后缀的图片
|
||||
kwargs:
|
||||
allowed_orig_suffix: # 后缀列表,表示只想下载以.gif结尾的图片
|
||||
- .gif
|
||||
|
||||
- plugin: client_proxy # 客户端实现类代理插件,不建议非开发人员使用
|
||||
kwargs:
|
||||
proxy_client_key: cl_proxy_future # 代理类的client_key
|
||||
whitelist: [ api, ] # 白名单,当client.impl匹配白名单时才代理
|
||||
|
||||
after_album:
|
||||
- plugin: zip # 压缩文件插件
|
||||
kwargs:
|
||||
level: photo # 按照章节,一个章节一个压缩文件
|
||||
filename_rule: Ptitle # 压缩文件的命名规则
|
||||
zip_dir: D:/jmcomic/zip/ # 压缩文件存放的文件夹
|
||||
delete_original_file: true # 压缩成功后,删除所有原文件和文件夹
|
||||
|
||||
```
|
@ -1,4 +1,4 @@
|
||||
# 教程:使用GitHub Actions下载禁漫本子
|
||||
# GitHub Actions教程
|
||||
|
||||
一共需要三步:
|
||||
|
||||
@ -18,31 +18,31 @@
|
||||
|
||||
(最新提示,下图的1可以不做,即直接点绿色的Create fork按钮)
|
||||
|
||||

|
||||

|
||||
|
||||
## 2. 填写你需要下载的本子id
|
||||
|
||||
### 2.1. 方式一(最新、推荐)
|
||||
### 2.1. 方式一(最新、简单、推荐)
|
||||
|
||||
访问下面这个网址:
|
||||
|
||||
`https://github.com/你的用户名/JMComic-Crawler-Python/actions/workflows/download_dispatch.yml`
|
||||
`https://github.com/你的用户名/JMComic-Crawler-Python/master/workflows/download_dispatch.yml`
|
||||
|
||||
按下图步骤进行操作:
|
||||
|
||||

|
||||

|
||||
|
||||
### 2.2. 方式二
|
||||
|
||||
访问下面这个网址:
|
||||
|
||||
`https://github.com/你的用户名/JMComic-Crawler-Python/edit/workflow/usage/workflow_download.py`
|
||||
`https://github.com/你的用户名/JMComic-Crawler-Python/edit/master/usage/workflow_download.py`
|
||||
|
||||
其实就是编辑`workflow`分支下的 `usage/workflow_download.py`文件
|
||||
其实就是编辑 `usage/workflow_download.py`文件,任意分支皆可。
|
||||
|
||||
按下图步骤进行操作:
|
||||
|
||||

|
||||

|
||||
|
||||
看到上面的绿色框里的代码了吗?(位置可能有些变化)
|
||||
|
||||
@ -54,11 +54,11 @@
|
||||
|
||||
来到Actions页面,选择最新的一次记录,等待它完成。
|
||||
|
||||

|
||||

|
||||
|
||||
完成以后,在页面最底部,点击下载你的成品即可。
|
||||
|
||||

|
||||

|
||||
|
||||
如果你发现GitHub Actions显示❌,表明出现了问题,运行失败。
|
||||
|
25
assets/docs/sources/tutorial/2_command_line.md
Normal file
@ -0,0 +1,25 @@
|
||||
# 命令行教程
|
||||
|
||||
## 1. 基本用法
|
||||
|
||||
```
|
||||
# 下载album 123 456,下载photo 333。彼此之间使用空格间隔
|
||||
jmcomic 123 456 p333
|
||||
```
|
||||
|
||||
## 2. 自定义option
|
||||
|
||||
### 2.1. 通过命令行
|
||||
使用 --option 参数指定option配置文件路径
|
||||
|
||||
```
|
||||
jmcomic 123 --option="D:/a.yml"
|
||||
```
|
||||
|
||||
### 2.2. 使用环境变量
|
||||
配置环境变量 `JM_OPTION_PATH` 为option配置文件路径
|
||||
|
||||
```
|
||||
set JM_OPTION_PATH="D:/a.yml"
|
||||
jmcomic 123
|
||||
```
|
84
assets/docs/sources/tutorial/3_demo.md
Normal file
@ -0,0 +1,84 @@
|
||||
# 使用jmcomic实现简单功能
|
||||
|
||||
## 下载本子
|
||||
|
||||
```python
|
||||
from jmcomic import *
|
||||
|
||||
ls = str_to_list('''
|
||||
438696
|
||||
https://18comic.vip/album/497896/
|
||||
''')
|
||||
|
||||
download_album(ls)
|
||||
```
|
||||
|
||||
## 获取实体类
|
||||
|
||||
```python
|
||||
from jmcomic import *
|
||||
|
||||
# 客户端
|
||||
client = JmOption.default().new_jm_client()
|
||||
|
||||
# 本子实体类
|
||||
album: JmAlbumDetail = client.get_album_detail('427413')
|
||||
|
||||
|
||||
def fetch(photo: JmPhotoDetail):
|
||||
# 章节实体类
|
||||
photo = client.get_photo_detail(photo.photo_id, False)
|
||||
|
||||
# 图片实体类
|
||||
image: JmImageDetail
|
||||
for image in photo:
|
||||
print(image.img_url)
|
||||
|
||||
# 多线程发起请求
|
||||
multi_thread_launcher(
|
||||
iter_objs=album,
|
||||
apply_each_obj_func=fetch
|
||||
)
|
||||
```
|
||||
|
||||
## 搜索本子
|
||||
|
||||
```python
|
||||
from jmcomic import *
|
||||
|
||||
client = JmOption.default().new_jm_client()
|
||||
|
||||
# 分页查询,search_site就是禁漫网页上的【站内搜索】
|
||||
page: JmSearchPage = client.search_site(search_query='+MANA +无修正', page=1)
|
||||
# page默认的迭代方式是page.iter_id_title(),每次迭代返回 albun_id, title
|
||||
for album_id, title in page:
|
||||
print(f'[{album_id}]: {title}')
|
||||
|
||||
# 直接搜索禁漫车号
|
||||
page = client.search_site(search_query='427413')
|
||||
album: JmAlbumDetail = page.single_album
|
||||
print(album.tags)
|
||||
```
|
||||
|
||||
## 搜索并下载本子
|
||||
|
||||
```python
|
||||
from jmcomic import *
|
||||
|
||||
option = JmOption.default()
|
||||
client = option.new_jm_client()
|
||||
|
||||
tag = '無修正'
|
||||
# 搜索标签,可以使用search_tag。
|
||||
# 搜索第一页。
|
||||
page: JmSearchPage = client.search_tag(tag, page=1)
|
||||
|
||||
aid_list = []
|
||||
|
||||
for aid, atitle, tag_list in page.iter_id_title_tag(): # 使用page的iter_id_title_tag迭代器
|
||||
if tag in tag_list:
|
||||
print(f'[标签/{tag}] 发现目标: [{aid}]: [{atitle}]')
|
||||
aid_list.append(aid)
|
||||
|
||||
download_album(aid_list, option)
|
||||
```
|
@ -1,13 +1,20 @@
|
||||
"""
|
||||
本文件演示对jmcomic模块进行自定义功能的方式,下面的每个函数都是一个独立的演示单元。
|
||||
本文件不演示【自定义配置】,有关配置的教程文档请见 ``
|
||||
"""
|
||||
from jmcomic import *
|
||||
# jmcomic 模块自定义
|
||||
|
||||
|
||||
|
||||
下方所有函数都省略了如下的导包和准备代码
|
||||
|
||||
```python
|
||||
from jmcomic import *
|
||||
option = JmOption.default()
|
||||
client: JmcomicClient = option.build_jm_client()
|
||||
```
|
||||
|
||||
|
||||
|
||||
## 自定义下载前后的回调函数
|
||||
|
||||
```python
|
||||
def custom_download_callback():
|
||||
"""
|
||||
该函数演示自定义下载时的回调函数
|
||||
@ -21,33 +28,43 @@ def custom_download_callback():
|
||||
def after_album(self, album: JmAlbumDetail):
|
||||
print(f'album下载完毕: {album}')
|
||||
pass
|
||||
|
||||
# 同样的,最后要让你的自定义类生效
|
||||
|
||||
# 最后,让你的自定义类生效
|
||||
JmModuleConfig.CLASS_DOWNLOADER = MyDownloader
|
||||
```
|
||||
|
||||
|
||||
|
||||
## 自定义option类
|
||||
|
||||
|
||||
```python
|
||||
def custom_option_class():
|
||||
"""
|
||||
该函数演示自定义option
|
||||
该函数演示自定义option类
|
||||
"""
|
||||
|
||||
# jmcomic模块支持自定义Option类,
|
||||
# 你可以写一个自己的类,继承JmOption,然后覆盖其中的一些方法。
|
||||
class MyOption(JmOption):
|
||||
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
print('MyOption 初始化开始')
|
||||
super().__init__(*args, **kwargs)
|
||||
|
||||
|
||||
@classmethod
|
||||
def default(cls):
|
||||
print('调用了MyOption.default()')
|
||||
return super().default()
|
||||
|
||||
|
||||
# 最后,替换默认Option类即可
|
||||
JmModuleConfig.CLASS_OPTION = MyOption
|
||||
```
|
||||
|
||||
|
||||
## 自定义client类
|
||||
|
||||
```python
|
||||
def custom_client_class():
|
||||
"""
|
||||
该文件演示自定义client类
|
||||
@ -55,68 +72,77 @@ def custom_client_class():
|
||||
|
||||
# 默认情况下,JmOption使用client类是根据配置项 `client.impl` 决定的
|
||||
# JmOption会根据`client.impl`到 JmModuleConfig.CLASS_CLIENT_IMPL 中查找
|
||||
|
||||
|
||||
# 自定义client的步骤如下
|
||||
|
||||
|
||||
# 1. 自定义Client类
|
||||
class MyClient(JmHtmlClient):
|
||||
client_key = 'myclient'
|
||||
pass
|
||||
|
||||
|
||||
# 2. 让MyClient生效
|
||||
JmModuleConfig.register_client(MyClient)
|
||||
|
||||
|
||||
# 3. 在配置文件中使用你定义的client.impl,后续使用这个option即可
|
||||
"""
|
||||
client:
|
||||
impl: myclient
|
||||
"""
|
||||
```
|
||||
|
||||
|
||||
## 自定义实体类(本子/章节/图片)
|
||||
|
||||
```python
|
||||
def custom_album_photo_image_detail_class():
|
||||
"""
|
||||
该函数演示替换实体类(本子/章节/图片)
|
||||
该函数演示自定义实体类(本子/章节/图片)
|
||||
|
||||
在使用路径规则 DirRule 时,可能会遇到需要自定义实体类属性的情况,例如:
|
||||
dir_rule:
|
||||
base_dir: ${workspace}
|
||||
rule: Bd_Acustom_Pcustom
|
||||
|
||||
|
||||
上面的Acustom,Pcustom都是自定义字段
|
||||
如果你想要使用这种自定义字段,你就需要替换默认的实体类,方式如下
|
||||
"""
|
||||
|
||||
|
||||
# 自定义本子实体类
|
||||
class MyAlbum(JmAlbumDetail):
|
||||
# 自定义 custom 属性
|
||||
@property
|
||||
def custom(self):
|
||||
return f'custom_{self.title}'
|
||||
|
||||
|
||||
# 自定义章节实体类
|
||||
class MyPhoto(JmPhotoDetail):
|
||||
# 自定义 custom 属性
|
||||
@property
|
||||
def custom(self):
|
||||
return f'custom_{self.title}'
|
||||
|
||||
|
||||
"""
|
||||
v2.3.3: 支持更灵活的自定义方式,可以使用函数,效果同上,示例见下
|
||||
"""
|
||||
|
||||
|
||||
class MyAlbum2(JmAlbumDetail):
|
||||
|
||||
|
||||
def get_dirname(self, ref: str) -> str:
|
||||
if ref == 'custom':
|
||||
return f'custom_{self.name}'
|
||||
|
||||
|
||||
return super().get_dirname(ref)
|
||||
|
||||
|
||||
# 最后,替换默认实体类来让你的自定义类生效
|
||||
JmModuleConfig.CLASS_ALBUM = MyAlbum
|
||||
JmModuleConfig.CLASS_PHOTO = MyPhoto
|
||||
```
|
||||
|
||||
|
||||
|
||||
## 自定义debug
|
||||
|
||||
```python
|
||||
def custom_jm_debug():
|
||||
"""
|
||||
该函数演示自定义debug
|
||||
@ -125,7 +151,7 @@ def custom_jm_debug():
|
||||
# jmcomic模块在运行过程中会使用 jm_debug() 这个函数进行打印信息
|
||||
# jm_debug() 这个函数 最后会调用 JmModuleConfig.debug_executor 函数
|
||||
# 你可以写一个自己的函数,替换 JmModuleConfig.debug_executor,实现自定义debug
|
||||
|
||||
|
||||
# 1. 自定义debug函数
|
||||
def my_debug(topic: str, msg: str):
|
||||
"""
|
||||
@ -134,6 +160,7 @@ def custom_jm_debug():
|
||||
@param msg: 具体debug的信息
|
||||
"""
|
||||
pass
|
||||
|
||||
|
||||
# 2. 让my_debug生效
|
||||
JmModuleConfig.debug_executor = my_debug
|
||||
```
|
@ -1,33 +1,33 @@
|
||||
"""
|
||||
# Filter - 下载过滤器
|
||||
|
||||
filter(过滤器)是v2.1.12新引入的机制,
|
||||
利用filter,你可以实现下载时过滤本子/章节/图片,完全控制你要下载的内容。
|
||||
|
||||
使用filter的步骤如下:
|
||||
|
||||
```
|
||||
1. 自定义class,继承JmDownloader,重写filter_iter_objs方法,即:
|
||||
class MyDownloader(JmDownloader):
|
||||
def filter_iter_objs(self, iter_objs: DownloadIterObjs):
|
||||
# 如何重写?参考JmDownloader.filter_iter_objs和下面的示例
|
||||
...
|
||||
class MyDownloader(JmDownloader):
|
||||
def filter_iter_objs(self, iter_objs: DownloadIterObjs):
|
||||
# 如何重写?参考JmDownloader.filter_iter_objs和下面的示例
|
||||
...
|
||||
|
||||
2. 让你的class生效,使用如下代码:
|
||||
JmModuleConfig.CLASS_DOWNLOADER = MyDownloader
|
||||
JmModuleConfig.CLASS_DOWNLOADER = MyDownloader
|
||||
|
||||
3. 照常使用下载api:
|
||||
download_album(xxx, option)
|
||||
download_album(xxx, option)
|
||||
```
|
||||
|
||||
** 本文件下面的示例只演示步骤1 **
|
||||
|
||||
本文件包含如下示例:
|
||||
- 只下载章节的前三张图
|
||||
- 只下载本子的特定章节以后的章节
|
||||
* 下面的示例只演示步骤1
|
||||
|
||||
|
||||
"""
|
||||
|
||||
## 示例1:只下载章节的前三张图
|
||||
|
||||
```python
|
||||
from jmcomic import *
|
||||
|
||||
|
||||
# 示例:只下载章节的前三张图
|
||||
class First3ImageDownloader(JmDownloader):
|
||||
|
||||
def filter_iter_objs(self, iter_objs: DownloadIterObjs):
|
||||
@ -37,9 +37,15 @@ class First3ImageDownloader(JmDownloader):
|
||||
return photo[:3]
|
||||
|
||||
return iter_objs
|
||||
```
|
||||
|
||||
|
||||
# 示例:只下载本子的特定章节以后的章节
|
||||
|
||||
## 示例2:只下载本子的特定章节以后的章节
|
||||
|
||||
```python
|
||||
from jmcomic import *
|
||||
|
||||
# 参考:https://github.com/hect0x7/JMComic-Crawler-Python/issues/95
|
||||
class FindUpdateDownloader(JmDownloader):
|
||||
album_after_photo = {
|
||||
@ -69,3 +75,4 @@ class FindUpdateDownloader(JmDownloader):
|
||||
is_new_photo = True
|
||||
|
||||
return photo_ls
|
||||
```
|
@ -1,46 +1,56 @@
|
||||
"""
|
||||
plugin(扩展/插件)是v2.2.0新引入的机制,
|
||||
plugin机制可以实现在`特定时间` 回调 `特定插件`,实现灵活无感知的功能增强,
|
||||
# Plugin - 插件
|
||||
|
||||
plugin(扩展/插件)是v2.2.0新引入的机制,使用插件可以实现灵活无感知的功能增强。
|
||||
|
||||
plugin的机制为,可在`特定时间` 回调 `特定插件`。
|
||||
|
||||
目前jmcomic已经内置了一些强大的插件,源码位于 src/jmcomic/jm_plugin.py。
|
||||
|
||||
配置插件的方式请查看 [option_file_syntax](../option_file_syntax.md#3-option插件配置项)
|
||||
|
||||
|
||||
|
||||
## 示例1. 使用login插件,自动登录禁漫
|
||||
|
||||
特定时间:after_init(option对象实例化完毕后)
|
||||
|
||||
下面举例演示:
|
||||
特定时间:after_init
|
||||
特定插件:login
|
||||
|
||||
插件功能:登录JM,获取并保存cookies,所有的Client都会使用这个cookies。
|
||||
|
||||
你可以在option配置文件当中,配置如下内容,来实现在after_init时,调用login插件。
|
||||
首先,你需要在option配置文件当中配置如下内容,来实现在after_init时,调用login插件。
|
||||
|
||||
plugin:
|
||||
```yml
|
||||
plugins:
|
||||
after_init: # 时机
|
||||
- plugin: login # 插件的key
|
||||
kwargs:
|
||||
# 下面是给插件的参数 (kwargs),由插件类自定义
|
||||
username: un # 禁漫帐号
|
||||
password: pw # 密码
|
||||
```
|
||||
|
||||
有了上述option配置,当你调用下面的代码时,login插件就会执行。
|
||||
|
||||
```python
|
||||
import jmcomic
|
||||
option = jmcomic.create_option('xxx.yml')
|
||||
# 程序走到这里,login插件已经调用完毕了。
|
||||
```
|
||||
|
||||
-----------------------------------------
|
||||
|
||||
目前jmcomic已经内置了一些强大的插件,源码位于 src/jmcomic/jm_plugin.py。
|
||||
配置插件的方式请查看 assets/config/option_plugin。
|
||||
|
||||
-----------------------------------------
|
||||
## 示例2. 自定义插件
|
||||
|
||||
你也可以自定义插件和插件时机
|
||||
自定义插件时机需要你重写Option类,示例请见 usage_custom
|
||||
下面演示自定义插件,分为3步:
|
||||
|
||||
1. 自定义plugin类
|
||||
2. 让plugin类生效
|
||||
3. 使用plugin的key
|
||||
|
||||
如果你有好的plugin想法,也欢迎向我提PR,将你的plugin内置到jmcomic模块中
|
||||
|
||||
"""
|
||||
* 如果你有好的plugin想法,也欢迎向我提PR,将你的plugin内置到jmcomic模块中
|
||||
|
||||
```python
|
||||
# 1. 自定义plugin类
|
||||
from jmcomic import JmOptionPlugin, JmModuleConfig, create_option
|
||||
|
||||
@ -53,19 +63,29 @@ class MyPlugin(JmOptionPlugin):
|
||||
def invoke(self, hello_plugin) -> None:
|
||||
print(hello_plugin)
|
||||
|
||||
|
||||
# 2. 让plugin类生效
|
||||
JmModuleConfig.register_plugin(MyPlugin)
|
||||
```
|
||||
|
||||
接下来,在option中配置如下内容来使用你的插件
|
||||
|
||||
```yml
|
||||
# 3. 在配置文件中使用plugin
|
||||
"""
|
||||
plugin:
|
||||
plugins:
|
||||
after_init: # 时机
|
||||
- plugin: myplugin # 插件的key
|
||||
kwargs:
|
||||
hello_plugin: this is my plugin invoke method's parameter # 你自定义的插件的参数
|
||||
"""
|
||||
```
|
||||
|
||||
# 当你使用上述配置文件创建option时,
|
||||
# 在option初始化完成后,你的plugin会被调用,控制台就会打印出 `this is my plugin invoke method's parameter`
|
||||
完成上述步骤后,每当你使用下面的代码,你的plugin会被调用,控制台就会打印出 `this is my plugin invoke method's parameter`
|
||||
|
||||
```python
|
||||
from jmcomic import create_option
|
||||
option = create_option('xxx')
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
54
assets/docs/sources/tutorial/7_advance.md
Normal file
@ -0,0 +1,54 @@
|
||||
# jmcomic 综合使用实例
|
||||
|
||||
* 基于v2.2.2+内置插件
|
||||
|
||||
## 功能需求
|
||||
|
||||
1. 下载id为145504的本子;
|
||||
2. 只下载章节在290266之后的新章;
|
||||
3. 最后要把图片转为jpg格式;
|
||||
4. 要使用登录状态去下载;
|
||||
5. 压缩文件,按照章节压缩,一个章节一个压缩文件,压缩文件的命名: 章节标题.zip;
|
||||
6. 自动把下载的文件全部删除,最后只要保留压缩文件。
|
||||
|
||||
## 实现方案
|
||||
|
||||
#### 1. 写2行python代码
|
||||
```python
|
||||
from jmcomic import create_option
|
||||
create_option('myoption.yml')
|
||||
```
|
||||
|
||||
#### 2. 配置option,调用1的脚本即可
|
||||
```yml
|
||||
dir_rule: # 下载路径规则
|
||||
rule: Bd_Aid
|
||||
base_dir: D:/jmcomic
|
||||
|
||||
download:
|
||||
image:
|
||||
suffix: .jpg # 转为jpg格式的图片
|
||||
|
||||
client:
|
||||
domain:
|
||||
- 18comic.vip # 指定域名
|
||||
|
||||
plugins:
|
||||
after_init:
|
||||
- plugin: login # 登录插件
|
||||
kwargs:
|
||||
username: un
|
||||
password: pw
|
||||
|
||||
- plugin: find_update # 只下载新章插件
|
||||
kwargs:
|
||||
145504: 290266 # 下载本子145504的章节290266以后的新章
|
||||
|
||||
after_album:
|
||||
- plugin: zip # 压缩文件插件
|
||||
kwargs:
|
||||
level: photo # 按照章节,一个章节一个压缩文件
|
||||
filename_rule: Ptitle # 压缩文件的命名规则
|
||||
zip_dir: D:/jmcomic # 压缩文件存放的文件夹
|
||||
delete_original_file: true # 压缩成功后,删除所有原文件和文件夹
|
||||
```
|
@ -1,6 +1,8 @@
|
||||
"""
|
||||
该脚本的作用:测试使用当前ip可以访问哪些禁漫域名
|
||||
# 趣味用法:测试你的ip可以访问哪些禁漫域名
|
||||
|
||||
```python
|
||||
"""
|
||||
该脚本的作用:测试当前ip可以访问哪些禁漫域名
|
||||
"""
|
||||
|
||||
from jmcomic import *
|
||||
@ -62,3 +64,6 @@ multi_thread_launcher(
|
||||
|
||||
for domain, status in domain_status_dict.items():
|
||||
print(f'{domain}: {status}')
|
||||
|
||||
```
|
||||
|
@ -15,7 +15,7 @@ client:
|
||||
- jmcomic.me
|
||||
|
||||
# 插件配置
|
||||
plugin:
|
||||
plugins:
|
||||
after_init:
|
||||
- plugin: usage_log # 实时打印硬件占用率的插件
|
||||
kwargs:
|
@ -15,7 +15,7 @@ client:
|
||||
- jmcomic.me
|
||||
|
||||
# 插件配置
|
||||
plugin:
|
||||
plugins:
|
||||
after_init:
|
||||
- plugin: usage_log # 实时打印硬件占用率的插件
|
||||
kwargs:
|
@ -8,7 +8,7 @@ client:
|
||||
html: [jmcomic1.me, jmcomic.me]
|
||||
|
||||
# 插件配置
|
||||
plugin:
|
||||
plugins:
|
||||
after_init:
|
||||
- plugin: usage_log # 实时打印硬件占用率的插件
|
||||
kwargs:
|
2
setup.py
@ -27,7 +27,7 @@ setup(
|
||||
package_dir={"": "src"},
|
||||
python_requires=">=3.7",
|
||||
install_requires=[
|
||||
'commonX>=0.5.6',
|
||||
'commonX>=0.5.7',
|
||||
'curl_cffi',
|
||||
'PyYAML',
|
||||
'Pillow',
|
||||
|
@ -2,7 +2,7 @@
|
||||
# 被依赖方 <--- 使用方
|
||||
# config <--- entity <--- toolkit <--- client <--- option <--- downloader
|
||||
|
||||
__version__ = '2.3.9'
|
||||
__version__ = '2.3.10'
|
||||
|
||||
from .api import *
|
||||
from .jm_plugin import *
|
||||
|
@ -8,12 +8,13 @@ def download_batch(download_api,
|
||||
):
|
||||
"""
|
||||
批量下载 album / photo
|
||||
|
||||
一个album/photo,对应一个线程,对应一个option
|
||||
|
||||
@param download_api: 下载api
|
||||
@param jm_id_iter: jmid (album_id, photo_id) 的迭代器
|
||||
@param option: 下载选项,对所有的jmid使用同一个,默认是 JmOption.default()
|
||||
@param downloader: 下载器类
|
||||
:param download_api: 下载api
|
||||
:param jm_id_iter: jmid (album_id, photo_id) 的迭代器
|
||||
:param option: 下载选项,所有的jmid共用一个option
|
||||
:param downloader: 下载器类
|
||||
"""
|
||||
from common import multi_thread_launcher
|
||||
|
||||
@ -31,11 +32,13 @@ def download_batch(download_api,
|
||||
|
||||
def download_album(jm_album_id, option=None, downloader=None):
|
||||
"""
|
||||
下载一个本子
|
||||
@param jm_album_id: 禁漫的本子的id,类型可以是str/int/iterable[str]。
|
||||
如果是iterable[str],则会调用 download_album_batch
|
||||
@param option: 下载选项,为空默认是 JmOption.default()
|
||||
@param downloader: 下载器类
|
||||
下载一个本子(album),包含其所有的章节(photo)
|
||||
|
||||
当jm_album_id不是str或int时,相当于调用 download_batch(download_album, jm_album_id, option, downloader)
|
||||
|
||||
:param jm_album_id: 本子的禁漫车号
|
||||
:param option: 下载选项
|
||||
:param downloader: 下载器类
|
||||
"""
|
||||
|
||||
if not isinstance(jm_album_id, (str, int)):
|
||||
@ -47,7 +50,7 @@ def download_album(jm_album_id, option=None, downloader=None):
|
||||
|
||||
def download_photo(jm_photo_id, option=None, downloader=None):
|
||||
"""
|
||||
下载一个章节
|
||||
下载一个章节(photo),参数同 download_album
|
||||
"""
|
||||
if not isinstance(jm_photo_id, (str, int)):
|
||||
return download_batch(download_photo, jm_photo_id, option)
|
||||
|
@ -61,12 +61,12 @@ class AbstractJmClient(
|
||||
):
|
||||
"""
|
||||
统一请求,支持重试
|
||||
@param request: 请求方法
|
||||
@param url: 图片url / path (/album/xxx)
|
||||
@param domain_index: 域名下标
|
||||
@param retry_count: 重试次数
|
||||
@param judge: 判定响应是否成功
|
||||
@param kwargs: 请求方法的kwargs
|
||||
:param request: 请求方法
|
||||
:param url: 图片url / path (/album/xxx)
|
||||
:param domain_index: 域名下标
|
||||
:param retry_count: 重试次数
|
||||
:param judge: 判定响应是否成功
|
||||
:param kwargs: 请求方法的kwargs
|
||||
"""
|
||||
if domain_index >= len(self.domain_list):
|
||||
self.fallback(request, url, domain_index, retry_count, **kwargs)
|
||||
@ -335,15 +335,15 @@ class JmHtmlClient(AbstractJmClient):
|
||||
return ret
|
||||
|
||||
@classmethod
|
||||
def require_resp_success_else_raise(cls, resp, org_req_url: str):
|
||||
def require_resp_success_else_raise(cls, resp, orig_req_url: str):
|
||||
"""
|
||||
@param resp: 响应对象
|
||||
@param org_req_url: /photo/12412312
|
||||
:param resp: 响应对象
|
||||
:param orig_req_url: /photo/12412312
|
||||
"""
|
||||
# 1. 检查是否 album_missing
|
||||
error_album_missing = '/error/album_missing'
|
||||
if resp.url.endswith(error_album_missing) and not org_req_url.endswith(error_album_missing):
|
||||
ExceptionTool.raise_missing(resp, org_req_url)
|
||||
if resp.url.endswith(error_album_missing) and not orig_req_url.endswith(error_album_missing):
|
||||
ExceptionTool.raise_missing(resp, orig_req_url)
|
||||
|
||||
# 2. 是否是特殊的内容
|
||||
cls.check_special_text(resp)
|
||||
@ -541,14 +541,14 @@ class JmApiClient(AbstractJmClient):
|
||||
return 'api'
|
||||
|
||||
@classmethod
|
||||
def require_resp_success(cls, resp: JmApiResp, org_req_url: str):
|
||||
def require_resp_success(cls, resp: JmApiResp, orig_req_url: str):
|
||||
resp.require_success()
|
||||
|
||||
# 1. 检查是否 album_missing
|
||||
# json: {'code': 200, 'data': []}
|
||||
data = resp.model().data
|
||||
if isinstance(data, list) and len(data) == 0:
|
||||
ExceptionTool.raise_missing(resp, org_req_url)
|
||||
ExceptionTool.raise_missing(resp, orig_req_url)
|
||||
|
||||
# 2. 是否是特殊的内容
|
||||
# 暂无
|
||||
@ -561,7 +561,7 @@ class FutureClientProxy(JmcomicClient):
|
||||
|
||||
可通过插件 ClientProxyPlugin 启用本类,配置如下:
|
||||
```yml
|
||||
plugin:
|
||||
plugins:
|
||||
after_init:
|
||||
- plugin: client_proxy
|
||||
kwargs:
|
||||
|
@ -176,7 +176,7 @@ class JmDetailClient:
|
||||
本方法会检查photo是不是[1],
|
||||
如果是[1],通过请求获取[2],然后把2中的一些重要字段更新到1中
|
||||
|
||||
@param photo: 被检查的JmPhotoDetail对象
|
||||
:param photo: 被检查的JmPhotoDetail对象
|
||||
"""
|
||||
# 检查 from_album
|
||||
if photo.from_album is None:
|
||||
@ -210,12 +210,12 @@ class JmUserClient:
|
||||
) -> JmAcResp:
|
||||
"""
|
||||
评论漫画/评论回复
|
||||
@param video_id: album_id/photo_id
|
||||
@param comment: 评论内容
|
||||
@param status: 是否 "有劇透"
|
||||
@param comment_id: 被回复评论的id
|
||||
@param originator:
|
||||
@return: JmAcResp 对象
|
||||
:param video_id: album_id/photo_id
|
||||
:param comment: 评论内容
|
||||
:param status: 是否 "有劇透"
|
||||
:param comment_id: 被回复评论的id
|
||||
:param originator:
|
||||
:returns: JmAcResp 对象
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
@ -232,10 +232,10 @@ class JmImageClient:
|
||||
):
|
||||
"""
|
||||
下载JM的图片
|
||||
@param img_url: 图片url
|
||||
@param img_save_path: 图片保存位置
|
||||
@param scramble_id: 图片所在photo的scramble_id
|
||||
@param decode_image: 要保存的是解密后的图还是原图
|
||||
:param img_url: 图片url
|
||||
:param img_save_path: 图片保存位置
|
||||
:param scramble_id: 图片所在photo的scramble_id
|
||||
:param decode_image: 要保存的是解密后的图还是原图
|
||||
"""
|
||||
if scramble_id is None:
|
||||
scramble_id = JmModuleConfig.SCRAMBLE_220980
|
||||
|
@ -101,7 +101,7 @@ class JmModuleConfig:
|
||||
# debug时解码url
|
||||
decode_url_when_debug = True
|
||||
# 下载时的一些默认值配置
|
||||
default_author = 'default-author'
|
||||
DEFAULT_AUTHOR = 'default-author'
|
||||
|
||||
@classmethod
|
||||
def downloader_class(cls):
|
||||
@ -169,7 +169,7 @@ class JmModuleConfig:
|
||||
def get_html_url(cls, postman=None):
|
||||
"""
|
||||
访问禁漫的永久网域,从而得到一个可用的禁漫网址
|
||||
@return: https://jm-comic2.cc
|
||||
:returns: https://jm-comic2.cc
|
||||
"""
|
||||
postman = postman or cls.new_postman(session=True)
|
||||
|
||||
@ -183,7 +183,7 @@ class JmModuleConfig:
|
||||
"""
|
||||
访问禁漫发布页,得到所有的禁漫网页域名
|
||||
|
||||
@return: ['18comic.vip', ..., 'jm365.xyz/ZNPJam'], 最后一个是【APP軟件下載】
|
||||
:returns: ['18comic.vip', ..., 'jm365.xyz/ZNPJam'], 最后一个是【APP軟件下載】
|
||||
"""
|
||||
postman = postman or cls.new_postman(session=True)
|
||||
|
||||
@ -232,6 +232,7 @@ class JmModuleConfig:
|
||||
def new_postman(cls, session=False, **kwargs):
|
||||
kwargs.setdefault('impersonate', 'chrome110')
|
||||
kwargs.setdefault('headers', JmModuleConfig.headers())
|
||||
kwargs.setdefault('proxies', JmModuleConfig.DEFAULT_PROXIES)
|
||||
return cls.postman_constructor(session, **kwargs)
|
||||
|
||||
album_comment_headers = {
|
||||
@ -256,11 +257,10 @@ class JmModuleConfig:
|
||||
|
||||
# option 相关的默认配置
|
||||
JM_OPTION_VER = '2.1'
|
||||
CLIENT_IMPL_DEFAULT = 'html'
|
||||
DEFAULT_CLIENT_IMPL = 'html'
|
||||
DEFAULT_PROXIES = system_proxy() # use system proxy by default
|
||||
|
||||
default_option_dict: dict = {
|
||||
'version': JM_OPTION_VER,
|
||||
'debug': None,
|
||||
'dir_rule': {'rule': 'Bd_Pname', 'base_dir': None},
|
||||
'download': {
|
||||
@ -285,7 +285,7 @@ class JmModuleConfig:
|
||||
'impl': None,
|
||||
'retry_times': 5
|
||||
},
|
||||
'plugin': {},
|
||||
'plugins': {},
|
||||
}
|
||||
|
||||
@classmethod
|
||||
@ -315,7 +315,7 @@ class JmModuleConfig:
|
||||
|
||||
# client impl
|
||||
if client['impl'] is None:
|
||||
client['impl'] = cls.CLIENT_IMPL_DEFAULT
|
||||
client['impl'] = cls.DEFAULT_CLIENT_IMPL
|
||||
|
||||
# postman proxies
|
||||
meta_data = client['postman']['meta_data']
|
||||
|
@ -97,6 +97,8 @@ class JmDownloader(DownloadCallback):
|
||||
|
||||
def download_by_image_detail(self, image: JmImageDetail, client: JmcomicClient):
|
||||
img_save_path = self.option.decide_image_filepath(image)
|
||||
|
||||
image.save_path = img_save_path
|
||||
image.is_exists = file_exists(img_save_path)
|
||||
|
||||
self.before_image(image, img_save_path)
|
||||
@ -153,8 +155,8 @@ class JmDownloader(DownloadCallback):
|
||||
只想下载 本子的最新一章,返回 [album[-1]]
|
||||
只想下载 章节的前10张图片,返回 [photo[:10]]
|
||||
|
||||
@param iter_objs: 可能是本子或者章节,需要自行使用 isinstance 判断
|
||||
@return: 只想要下载的 本子的章节 或 章节的图片
|
||||
:param iter_objs: 可能是本子或者章节,需要自行使用 isinstance 判断
|
||||
:returns: 只想要下载的 本子的章节 或 章节的图片
|
||||
"""
|
||||
return iter_objs
|
||||
|
||||
|
@ -64,8 +64,8 @@ class DetailEntity(JmBaseEntity, IndexedEntity):
|
||||
|
||||
用户可重写此方法,来实现自定义文件夹名
|
||||
|
||||
@param ref: 字段名
|
||||
@return: 文件夹名
|
||||
:param ref: 字段名
|
||||
:returns: 文件夹名
|
||||
"""
|
||||
return getattr(self, ref)
|
||||
|
||||
@ -94,9 +94,12 @@ class JmImageDetail(JmBaseEntity):
|
||||
|
||||
self.from_photo: Optional[JmPhotoDetail] = from_photo
|
||||
self.query_params: StrNone = query_params
|
||||
self.is_exists: bool = False
|
||||
self.index = index
|
||||
|
||||
# temp fields, in order to simplify passing parameter
|
||||
self.save_path: str = ''
|
||||
self.is_exists: bool = False
|
||||
|
||||
@property
|
||||
def filename_without_suffix(self):
|
||||
return self.img_file_name
|
||||
@ -110,7 +113,7 @@ class JmImageDetail(JmBaseEntity):
|
||||
"""
|
||||
图片的下载路径
|
||||
与 self.img_url 的唯一不同是,在最后会带上 ?{self.query_params}
|
||||
@return: 图片的下载路径
|
||||
:returns: 图片的下载路径
|
||||
"""
|
||||
if self.query_params is None:
|
||||
return self.img_url
|
||||
@ -250,7 +253,7 @@ class JmPhotoDetail(DetailEntity):
|
||||
return self._author.strip()
|
||||
|
||||
# 使用默认
|
||||
return JmModuleConfig.default_author
|
||||
return JmModuleConfig.DEFAULT_AUTHOR
|
||||
|
||||
def create_image_detail(self, index) -> JmImageDetail:
|
||||
# 校验参数
|
||||
@ -364,7 +367,7 @@ class JmAlbumDetail(DetailEntity):
|
||||
if len(self.authors) >= 1:
|
||||
return self.authors[0]
|
||||
|
||||
return JmModuleConfig.default_author
|
||||
return JmModuleConfig.DEFAULT_AUTHOR
|
||||
|
||||
@property
|
||||
def id(self):
|
||||
|
@ -92,10 +92,10 @@ class DirRule:
|
||||
"""
|
||||
应用规则解析器(RuleSolver)
|
||||
|
||||
@param album: JmAlbumDetail
|
||||
@param photo: JmPhotoDetail
|
||||
@param rule_solver: Ptitle
|
||||
@return: photo.title
|
||||
:param album: JmAlbumDetail
|
||||
:param photo: JmPhotoDetail
|
||||
:param rule_solver: Ptitle
|
||||
:returns: photo.title
|
||||
"""
|
||||
|
||||
def choose_detail(key):
|
||||
@ -121,7 +121,7 @@ class JmOption:
|
||||
dir_rule: Dict,
|
||||
download: Dict,
|
||||
client: Dict,
|
||||
plugin: Dict,
|
||||
plugins: Dict,
|
||||
filepath=None,
|
||||
):
|
||||
# 版本号
|
||||
@ -133,7 +133,7 @@ class JmOption:
|
||||
# 下载配置
|
||||
self.download = AdvancedEasyAccessDict(download)
|
||||
# 插件配置
|
||||
self.plugin = AdvancedEasyAccessDict(plugin)
|
||||
self.plugins = AdvancedEasyAccessDict(plugins)
|
||||
# 其他配置
|
||||
self.filepath = filepath
|
||||
|
||||
@ -220,8 +220,8 @@ class JmOption:
|
||||
使用默认的 JmOption
|
||||
proxies, domain 为常用配置项,为了方便起见直接支持参数配置。
|
||||
其他配置项建议还是使用配置文件
|
||||
@param proxies: clash; 127.0.0.1:7890; v2ray
|
||||
@param domain: 18comic.vip; ["18comic.vip"]
|
||||
:param proxies: clash; 127.0.0.1:7890; v2ray
|
||||
:param domain: 18comic.vip; ["18comic.vip"]
|
||||
"""
|
||||
if proxies is not None or domain is not None:
|
||||
return cls.construct({
|
||||
@ -234,8 +234,8 @@ class JmOption:
|
||||
return cls.construct({})
|
||||
|
||||
@classmethod
|
||||
def construct(cls, orgdic: Dict, cover_default=True) -> 'JmOption':
|
||||
dic = cls.merge_default_dict(orgdic) if cover_default else orgdic
|
||||
def construct(cls, origdic: Dict, cover_default=True) -> 'JmOption':
|
||||
dic = cls.merge_default_dict(origdic) if cover_default else origdic
|
||||
|
||||
# debug
|
||||
debug = dic.pop('debug', True)
|
||||
@ -244,18 +244,29 @@ class JmOption:
|
||||
|
||||
# version
|
||||
version = dic.pop('version', None)
|
||||
if version is None or float(version) >= float(JmModuleConfig.JM_OPTION_VER):
|
||||
if version is not None and float(version) >= float(JmModuleConfig.JM_OPTION_VER):
|
||||
# 版本号更高,跳过兼容代码
|
||||
return cls(**dic)
|
||||
|
||||
# 旧版本option,做兼容
|
||||
cls.compatible_with_old_versions(dic)
|
||||
|
||||
# 1) 2.0 -> 2.1,并发配置的键名更改了
|
||||
return cls(**dic)
|
||||
|
||||
@classmethod
|
||||
def compatible_with_old_versions(cls, dic):
|
||||
"""
|
||||
兼容旧的option版本
|
||||
"""
|
||||
# 1: 并发配置项
|
||||
dt: dict = dic['download']['threading']
|
||||
if 'batch_count' in dt:
|
||||
batch_count = dt.pop('batch_count')
|
||||
dt['image'] = batch_count
|
||||
|
||||
return cls(**dic)
|
||||
# 2: 插件配置项 plugin -> plugins
|
||||
if 'plugin' in dic:
|
||||
dic['plugins'] = dic.pop('plugin')
|
||||
|
||||
def deconstruct(self) -> Dict:
|
||||
return {
|
||||
@ -267,6 +278,7 @@ class JmOption:
|
||||
},
|
||||
'download': self.download.src_dict,
|
||||
'client': self.client.src_dict,
|
||||
'plugins': self.plugins.src_dict
|
||||
}
|
||||
|
||||
"""
|
||||
@ -399,7 +411,7 @@ class JmOption:
|
||||
# 下面的方法为调用插件提供支持
|
||||
|
||||
def call_all_plugin(self, group: str, **extra):
|
||||
plugin_list: List[dict] = self.plugin.get(group, [])
|
||||
plugin_list: List[dict] = self.plugins.get(group, [])
|
||||
if plugin_list is None or len(plugin_list) == 0:
|
||||
return
|
||||
|
||||
|
@ -14,7 +14,7 @@ class JmOptionPlugin:
|
||||
def invoke(self, **kwargs) -> None:
|
||||
"""
|
||||
执行插件的功能
|
||||
@param kwargs: 给插件的参数
|
||||
:param kwargs: 给插件的参数
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
@ -22,7 +22,7 @@ class JmOptionPlugin:
|
||||
def build(cls, option: JmOption) -> 'JmOptionPlugin':
|
||||
"""
|
||||
创建插件实例
|
||||
@param option: JmOption对象
|
||||
:param option: JmOption对象
|
||||
"""
|
||||
return cls(option)
|
||||
|
||||
@ -245,7 +245,7 @@ class ZipPlugin(JmOptionPlugin):
|
||||
def zip_photo(self, photo, image_list: list, zip_path: str):
|
||||
"""
|
||||
压缩photo文件夹
|
||||
@return: photo文件夹路径
|
||||
:returns: photo文件夹路径
|
||||
"""
|
||||
photo_dir = self.option.decide_image_save_dir(photo) \
|
||||
if len(image_list) == 0 \
|
||||
@ -265,7 +265,7 @@ class ZipPlugin(JmOptionPlugin):
|
||||
def zip_album(self, album, photo_dict: dict, zip_path):
|
||||
"""
|
||||
压缩album文件夹
|
||||
@return: album文件夹路径
|
||||
:returns: album文件夹路径
|
||||
"""
|
||||
album_dir = self.option.decide_album_dir(album)
|
||||
all_filepath: Set[str] = set()
|
||||
@ -351,3 +351,31 @@ class ClientProxyPlugin(JmOptionPlugin):
|
||||
return clazz(client, **clazz_init_kwargs)
|
||||
|
||||
self.option.new_jm_client = hook_new_jm_client
|
||||
|
||||
|
||||
class ImageSuffixFilterPlugin(JmOptionPlugin):
|
||||
plugin_key = 'image_suffix_filter'
|
||||
|
||||
def invoke(self,
|
||||
allowed_orig_suffix=None,
|
||||
) -> None:
|
||||
if allowed_orig_suffix is None:
|
||||
return
|
||||
|
||||
allowed_suffix_set = set(fix_suffix(suffix) for suffix in allowed_orig_suffix)
|
||||
|
||||
option_decide_cache = self.option.decide_download_cache
|
||||
|
||||
def apply_filter_then_decide_cache(image: JmImageDetail):
|
||||
if image.img_file_suffix not in allowed_suffix_set:
|
||||
jm_debug('image.filter.skip',
|
||||
f'跳过下载图片: {image.tag},'
|
||||
f'因为其后缀\'{image.img_file_suffix}\'不在允许的后缀集合{allowed_suffix_set}内')
|
||||
# hook is_exists True to skip download
|
||||
image.is_exists = True
|
||||
return True
|
||||
|
||||
# let option decide
|
||||
return option_decide_cache(image)
|
||||
|
||||
self.option.decide_download_cache = apply_filter_then_decide_cache
|
||||
|
@ -4,56 +4,56 @@ from .jm_entity import *
|
||||
|
||||
|
||||
class JmcomicText:
|
||||
pattern_jm_domain = compile('https://([\w.-]+)')
|
||||
pattern_jm_pa_id = compile('(photos?|album)/(\d+)')
|
||||
pattern_html_jm_pub_domain = compile('[\w-]+\.\w+/?\w+')
|
||||
pattern_jm_domain = compile(r'https://([\w.-]+)')
|
||||
pattern_jm_pa_id = compile(r'(photos?|album)/(\d+)')
|
||||
pattern_html_jm_pub_domain = compile(r'[\w-]+\.\w+/?\w+')
|
||||
|
||||
pattern_html_photo_photo_id = compile('<meta property="og:url" content=".*?/photo/(\d+)/?.*?">')
|
||||
pattern_html_photo_scramble_id = compile('var scramble_id = (\d+);')
|
||||
pattern_html_photo_name = compile('<title>([\s\S]*?)\|.*</title>')
|
||||
# pattern_html_photo_data_original_list = compile('data-original="(.*?)" id="album_photo_.+?"')
|
||||
pattern_html_photo_data_original_domain = compile('src="https://(.*?)/media/albums/blank')
|
||||
pattern_html_photo_data_original_0 = compile('data-original="(.*?)"[^>]*?id="album_photo[^>]*?data-page="0"')
|
||||
pattern_html_photo_tags = compile('<meta name="keywords"[\s\S]*?content="(.*?)"')
|
||||
pattern_html_photo_series_id = compile('var series_id = (\d+);')
|
||||
pattern_html_photo_sort = compile('var sort = (\d+);')
|
||||
pattern_html_photo_page_arr = compile('var page_arr = (.*?);')
|
||||
pattern_html_photo_photo_id = compile(r'<meta property="og:url" content=".*?/photo/(\d+)/?.*?">')
|
||||
pattern_html_photo_scramble_id = compile(r'var scramble_id = (\d+);')
|
||||
pattern_html_photo_name = compile(r'<title>([\s\S]*?)\|.*</title>')
|
||||
# pattern_html_photo_data_original_list = compile(r'data-original="(.*?)" id="album_photo_.+?"')
|
||||
pattern_html_photo_data_original_domain = compile(r'src="https://(.*?)/media/albums/blank')
|
||||
pattern_html_photo_data_original_0 = compile(r'data-original="(.*?)"[^>]*?id="album_photo[^>]*?data-page="0"')
|
||||
pattern_html_photo_tags = compile(r'<meta name="keywords"[\s\S]*?content="(.*?)"')
|
||||
pattern_html_photo_series_id = compile(r'var series_id = (\d+);')
|
||||
pattern_html_photo_sort = compile(r'var sort = (\d+);')
|
||||
pattern_html_photo_page_arr = compile(r'var page_arr = (.*?);')
|
||||
|
||||
pattern_html_album_album_id = compile('<span class="number">.*?:JM(\d+)</span>')
|
||||
pattern_html_album_scramble_id = compile('var scramble_id = (\d+);')
|
||||
pattern_html_album_name = compile('<h1 class="book-name" id="book-name">([\s\S]*?)</h1>')
|
||||
pattern_html_album_episode_list = compile('data-album="(\d+)">\n *?<li.*?>\n *'
|
||||
'第(\d+)話\n([\s\S]*?)\n *'
|
||||
'<[\s\S]*?>(\d+-\d+-\d+).*?')
|
||||
pattern_html_album_page_count = compile('<span class="pagecount">.*?:(\d+)</span>')
|
||||
pattern_html_album_pub_date = compile('>上架日期 : (.*?)</span>')
|
||||
pattern_html_album_update_date = compile('>更新日期 : (.*?)</span>')
|
||||
pattern_html_album_album_id = compile(r'<span class="number">.*?:JM(\d+)</span>')
|
||||
pattern_html_album_scramble_id = compile(r'var scramble_id = (\d+);')
|
||||
pattern_html_album_name = compile(r'<h1 class="book-name" id="book-name">([\s\S]*?)</h1>')
|
||||
pattern_html_album_episode_list = compile(r'data-album="(\d+)">\n *?<li.*?>\n *'
|
||||
r'第(\d+)話\n([\s\S]*?)\n *'
|
||||
r'<[\s\S]*?>(\d+-\d+-\d+).*?')
|
||||
pattern_html_album_page_count = compile(r'<span class="pagecount">.*?:(\d+)</span>')
|
||||
pattern_html_album_pub_date = compile(r'>上架日期 : (.*?)</span>')
|
||||
pattern_html_album_update_date = compile(r'>更新日期 : (.*?)</span>')
|
||||
# 作品
|
||||
pattern_html_album_works = [
|
||||
compile('<span itemprop="author" data-type="works">([\s\S]*?)</span>'),
|
||||
compile('<a[^>]*?>(.*?)</a>')
|
||||
compile(r'<span itemprop="author" data-type="works">([\s\S]*?)</span>'),
|
||||
compile(r'<a[^>]*?>(.*?)</a>')
|
||||
]
|
||||
# 登場人物
|
||||
pattern_html_album_actors = [
|
||||
compile('<span itemprop="author" data-type="actor">([\s\S]*?)</span>'),
|
||||
compile('<a[^>]*?>(.*?)</a>')
|
||||
compile(r'<span itemprop="author" data-type="actor">([\s\S]*?)</span>'),
|
||||
compile(r'<a[^>]*?>(.*?)</a>')
|
||||
]
|
||||
# 标签
|
||||
pattern_html_album_tags = [
|
||||
compile('<span itemprop="genre" data-type="tags">([\s\S]*?)</span>'),
|
||||
compile('<a[^>]*?>(.*?)</a>')
|
||||
compile(r'<span itemprop="genre" data-type="tags">([\s\S]*?)</span>'),
|
||||
compile(r'<a[^>]*?>(.*?)</a>')
|
||||
]
|
||||
# 作者
|
||||
pattern_html_album_authors = [
|
||||
compile('作者: *<span itemprop="author" data-type="author">([\s\S]*?)</span>'),
|
||||
compile("<a[^>]*?>(.*?)</a>"),
|
||||
compile(r'作者: *<span itemprop="author" data-type="author">([\s\S]*?)</span>'),
|
||||
compile(r"<a[^>]*?>(.*?)</a>"),
|
||||
]
|
||||
# 點擊喜歡
|
||||
pattern_html_album_likes = compile('<span id="albim_likes_\d+">(.*?)</span>')
|
||||
pattern_html_album_likes = compile(r'<span id="albim_likes_\d+">(.*?)</span>')
|
||||
# 觀看
|
||||
pattern_html_album_views = compile('<span>(.*?)</span> (次觀看|观看次数)')
|
||||
pattern_html_album_views = compile(r'<span>(.*?)</span> (次觀看|观看次数)')
|
||||
# 評論(div)
|
||||
pattern_html_album_comment_count = compile('<div class="badge"[^>]*?id="total_video_comments">(\d+)</div>'), 0
|
||||
pattern_html_album_comment_count = compile(r'<div class="badge"[^>]*?id="total_video_comments">(\d+)</div>'), 0
|
||||
|
||||
@classmethod
|
||||
def parse_to_jm_domain(cls, text: str):
|
||||
@ -214,29 +214,29 @@ class JmcomicText:
|
||||
|
||||
|
||||
# 支持dsl: #{???} -> os.getenv(???)
|
||||
JmcomicText.dsl_replacer.add_dsl_and_replacer('\$\{(.*?)\}', JmcomicText.match_os_env)
|
||||
JmcomicText.dsl_replacer.add_dsl_and_replacer(r'\$\{(.*?)\}', JmcomicText.match_os_env)
|
||||
|
||||
|
||||
class JmcomicSearchTool:
|
||||
# 用来缩减html的长度
|
||||
pattern_html_search_shorten_for = compile('<div class="well well-sm">([\s\S]*)<div class="row">')
|
||||
pattern_html_search_shorten_for = compile(r'<div class="well well-sm">([\s\S]*)<div class="row">')
|
||||
|
||||
# 用来提取搜索页面的的album的信息
|
||||
pattern_html_search_album_info_list = compile(
|
||||
'<a href="/album/(\d+)/.+"[\s\S]*?'
|
||||
'title="(.*?)"[\s\S]*?'
|
||||
'(<div class="label-category" style="">'
|
||||
'\n(.*)\n</div>\n<div class="label-sub" style=" ">'
|
||||
'(.*?)\n<[\s\S]*?)?'
|
||||
'<div class="title-truncate tags .*>\n'
|
||||
'(<a[\s\S]*?) </div>'
|
||||
r'<a href="/album/(\d+)/.+"[\s\S]*?'
|
||||
r'title="(.*?)"[\s\S]*?'
|
||||
r'(<div class="label-category" style="">'
|
||||
r'\n(.*)\n</div>\n<div class="label-sub" style=" ">'
|
||||
r'(.*?)\n<[\s\S]*?)?'
|
||||
r'<div class="title-truncate tags .*>\n'
|
||||
r'(<a[\s\S]*?) </div>'
|
||||
)
|
||||
|
||||
# 用来查找tag列表
|
||||
pattern_html_search_tag_list = compile('<a href=".*?">(.*?)</a>')
|
||||
pattern_html_search_tag_list = compile(r'<a href=".*?">(.*?)</a>')
|
||||
|
||||
# 查找错误,例如 [错误,關鍵字過短,請至少輸入兩個字以上。]
|
||||
pattern_html_search_error = compile('<fieldset>\n<legend>(.*?)</legend>\n<div class=.*?>\n(.*?)\n</div>\n</fieldset>')
|
||||
pattern_html_search_error = compile(r'<fieldset>\n<legend>(.*?)</legend>\n<div class=.*?>\n(.*?)\n</div>\n</fieldset>')
|
||||
|
||||
@classmethod
|
||||
def parse_html_to_page(cls, html: str) -> JmSearchPage:
|
||||
@ -464,9 +464,9 @@ class JmImageTool:
|
||||
如果需要改变图片的文件格式,比如 .jpg → .png,则需要指定参数 neet_convert=True.
|
||||
如果不需要改变图片的文件格式,使用 need_convert=False,可以跳过PIL解析图片,效率更高.
|
||||
|
||||
@param resp: HTTP响应对象
|
||||
@param filepath: 图片文件路径
|
||||
@param need_convert: 是否转换图片
|
||||
:param resp: HTTP响应对象
|
||||
:param filepath: 图片文件路径
|
||||
:param need_convert: 是否转换图片
|
||||
"""
|
||||
if need_convert is False:
|
||||
cls.save_directly(resp, filepath)
|
||||
@ -478,8 +478,8 @@ class JmImageTool:
|
||||
"""
|
||||
保存图片
|
||||
|
||||
@param image: PIL.Image对象
|
||||
@param filepath: 保存文件路径
|
||||
:param image: PIL.Image对象
|
||||
:param filepath: 保存文件路径
|
||||
"""
|
||||
image.save(filepath)
|
||||
|
||||
@ -496,9 +496,9 @@ class JmImageTool:
|
||||
) -> None:
|
||||
"""
|
||||
解密图片并保存
|
||||
@param num: 分割数,可以用 cls.calculate_segmentation_num 计算
|
||||
@param img_src: 原始图片
|
||||
@param decoded_save_path: 解密图片的保存路径
|
||||
:param num: 分割数,可以用 cls.calculate_segmentation_num 计算
|
||||
:param img_src: 原始图片
|
||||
:param decoded_save_path: 解密图片的保存路径
|
||||
"""
|
||||
|
||||
# 无需解密,直接保存
|
||||
@ -625,19 +625,19 @@ class ExceptionTool:
|
||||
@classmethod
|
||||
def raise_missing(cls,
|
||||
resp,
|
||||
org_req_url=None,
|
||||
orig_req_url=None,
|
||||
):
|
||||
"""
|
||||
抛出本子/章节的异常
|
||||
@param resp: 响应对象
|
||||
@param org_req_url: 原始请求url,可不传
|
||||
:param resp: 响应对象
|
||||
:param orig_req_url: 原始请求url,可不传
|
||||
"""
|
||||
if org_req_url is None:
|
||||
org_req_url = resp.url
|
||||
if orig_req_url is None:
|
||||
orig_req_url = resp.url
|
||||
|
||||
req_type = "本子" if "album" in org_req_url else "章节"
|
||||
req_type = "本子" if "album" in orig_req_url else "章节"
|
||||
cls.raises_resp((
|
||||
f'请求的{req_type}不存在!({org_req_url})\n'
|
||||
f'请求的{req_type}不存在!({orig_req_url})\n'
|
||||
'原因可能为:\n'
|
||||
f'1. id有误,检查你的{req_type}id\n'
|
||||
'2. 该漫画只对登录用户可见,请配置你的cookies,或者使用移动端Client(api)\n'
|
||||
|
@ -49,7 +49,7 @@ class JmTestConfigurable(unittest.TestCase):
|
||||
try:
|
||||
option = create_option_by_env('JM_OPTION_PATH_TEST')
|
||||
except JmcomicException:
|
||||
option = create_option('./assets/config/option_test.yml')
|
||||
option = create_option('./assets/option/option_test.yml')
|
||||
|
||||
cls.option = option
|
||||
cls.client = option.build_jm_client()
|
||||
@ -69,10 +69,6 @@ class JmTestConfigurable(unittest.TestCase):
|
||||
end = ts()
|
||||
print_eye_catching(f' [total cost {end - begin:.02f}s | {cls.__name__}] '.center(60, '-'))
|
||||
|
||||
@classmethod
|
||||
def use_option(cls, op_filename: str) -> JmOption:
|
||||
return create_option(f'./assets/config/{op_filename}')
|
||||
|
||||
@classmethod
|
||||
def adapt_os(cls):
|
||||
adapt_func_dict = {
|
||||
|
@ -1,91 +0,0 @@
|
||||
"""
|
||||
--------------------
|
||||
API快速上手
|
||||
--------------------
|
||||
"""
|
||||
import jmcomic # 导入此模块,需要先安装.
|
||||
|
||||
jmcomic.download_album('422866') # 传入要下载的album的id,即可下载整个album到本地.
|
||||
# 上面的这行代码,还有一个可选参数option: JmOption,表示配置项,
|
||||
# 配置项的作用是告诉程序下载时候的一些选择,
|
||||
# 比如,要下载到哪个文件夹,使用怎样的路径组织方式(比如[/作者/本子id/图片] 或者 [/作者/本子名称/图片]).
|
||||
# 如果没有配置,则会使用 JmOption.default(),下载的路径是[当前工作文件夹/本子章节名称/图片].
|
||||
|
||||
|
||||
"""
|
||||
--------------------
|
||||
批量下载介绍
|
||||
--------------------
|
||||
"""
|
||||
from jmcomic import download_album
|
||||
|
||||
# 如果你想要批量下载,可以使用 list/set/tuple/生成器 作为第一个参数。
|
||||
# 第二个参数依然是可选的JmOption对象
|
||||
download_album(['422866', '1', '2', '3']) # list
|
||||
download_album({'422866', '1', '2', '3'}) # set
|
||||
download_album(('422866', '1', '2', '3')) # tuple
|
||||
download_album(aid for aid in ('422866', '1', '2', '3')) # 生成器
|
||||
|
||||
|
||||
"""
|
||||
--------------------
|
||||
配置文件上手
|
||||
--------------------
|
||||
"""
|
||||
from jmcomic import JmModuleConfig, JmOption, create_option
|
||||
|
||||
# 先获取默认的JmOption对象
|
||||
option = JmOption.default()
|
||||
|
||||
# 可以把对象保存为文件,方便编辑
|
||||
option.to_file('保存路径.yml') # yml格式
|
||||
option.to_file('保存路径.json') # json格式
|
||||
|
||||
# 如果你修改了默认配置,现在想用你修改后的配置来下载,使用如下代码
|
||||
option = create_option('保存路径.yml')
|
||||
# 或者
|
||||
option = JmOption.from_file('保存路径.yml')
|
||||
# 使用你的option配置来下载
|
||||
jmcomic.download_album('23333', option)
|
||||
|
||||
|
||||
"""
|
||||
|
||||
--------------------
|
||||
配置文件进阶
|
||||
--------------------
|
||||
|
||||
定制option有几种方式?
|
||||
|
||||
方式1. 针对一个option对象生效
|
||||
JmOption.construct({xxx: yyy}) -- 简单配置推荐
|
||||
JmOption.from_file('配置文件路径') -- 复杂配置推荐
|
||||
|
||||
方式2. 针对所有option对象生效,全局配置,配置的优先级次于1
|
||||
JmModuleConfig.default_option_dict['xxx'] = yyy
|
||||
|
||||
下面以配置代理为例
|
||||
|
||||
"""
|
||||
from jmcomic import JmOption, ProxyBuilder, download_album
|
||||
|
||||
# 方式1. 定制一个option对象
|
||||
option = JmOption.construct({
|
||||
'client': {
|
||||
'postman': {
|
||||
'meta_data': {
|
||||
'proxies': ProxyBuilder.clash_proxy(),
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
# 调用下载api需要传入此option
|
||||
download_album('xxx', option)
|
||||
|
||||
# 方式2. 使用全局配置
|
||||
JmModuleConfig.default_option_dict['client']['postman']['meta_data']['proxies'] = ProxyBuilder.clash_proxy()
|
||||
# v2.3.9 以后,支持更简便的代理配置,且不配时默认使用系统代理:
|
||||
JmModuleConfig.DEFAULT_PROXIES = ProxyBuilder.clash_proxy()
|
||||
|
||||
# 调用下载api**不需要**传入option
|
||||
download_album('xxx')
|
@ -1,201 +0,0 @@
|
||||
"""
|
||||
jmcomic综合使用实例,基于v2.2.2+内置插件
|
||||
|
||||
功能需求:
|
||||
1. 下载id为145504的本子;
|
||||
2. 只下载章节在290266之后的新章;
|
||||
3. 最后要把图片转为jpg格式;
|
||||
4. 要使用登录状态去下载;
|
||||
5. 压缩文件,按照章节压缩,一个章节一个压缩文件,压缩文件的命名: 章节标题.zip;
|
||||
6. 自动把下载的文件全部删除,最后只要保留压缩文件。
|
||||
|
||||
|
||||
实现方案:
|
||||
1. 写2行python代码
|
||||
2. 配置option,调用1的脚本即可
|
||||
|
||||
"""
|
||||
|
||||
# 1. 写2行python代码
|
||||
from jmcomic import create_option
|
||||
create_option('myoption.yml')
|
||||
|
||||
# 2. 配置option,调用1的脚本即可
|
||||
"""
|
||||
dir_rule: # 下载路径规则
|
||||
rule: Bd_Aid
|
||||
base_dir: D:/jmcomic
|
||||
|
||||
download:
|
||||
image:
|
||||
suffix: .jpg # 转为jpg格式的图片
|
||||
|
||||
client:
|
||||
domain:
|
||||
- 18comic.vip # 指定域名
|
||||
|
||||
plugin:
|
||||
after_init:
|
||||
- plugin: login # 登录插件
|
||||
kwargs:
|
||||
username: un
|
||||
password: pw
|
||||
|
||||
- plugin: find_update # 只下载新章插件
|
||||
kwargs:
|
||||
145504: 290266 # 下载本子145504的章节290266以后的新章
|
||||
|
||||
after_album:
|
||||
- plugin: zip # 压缩文件插件
|
||||
kwargs:
|
||||
level: photo # 按照章节,一个章节一个压缩文件
|
||||
filename_rule: Ptitle # 压缩文件的命名规则
|
||||
zip_dir: D:/jmcomic # 压缩文件存放的文件夹
|
||||
delete_original_file: true # 压缩成功后,删除所有原文件和文件夹
|
||||
"""
|
||||
|
||||
|
||||
# 到此,程序已开始运行,最后会完成所有需求,并打印出如下日志。
|
||||
# 你可以详细查看以下日志,运行时所有关键节点都有日志打印。
|
||||
"""
|
||||
2023-08-01 02:23:57:【plugin.invoke】调用插件: [login]
|
||||
2023-08-01 02:23:57:【html】https://18comic.vip/login
|
||||
2023-08-01 02:23:58:【plugin.login】登录成功
|
||||
2023-08-01 02:23:58:【plugin.kwargs】插件参数类型转换: 145504 (<class 'int'>) -> 145504 (<class 'str'>)
|
||||
2023-08-01 02:23:58:【plugin.invoke】调用插件: [find_update]
|
||||
2023-08-01 02:23:58:【html】https://18comic.vip/album/145504
|
||||
2023-08-01 02:23:59:【album.before】本子获取成功: [145504], 作者: [G.HO], 章节数: [104], 总页数: [3598], 标题: [健身教练 / もしも、幼馴染を抱いたなら / Fitness], 关键词: [['完結', '韩漫', '上柱香再走', '以晨:我爹呢', '你妈真香', '連載中', '肌肉豪名不虚传']],
|
||||
2023-08-01 02:23:59:【html】https://18comic.vip/photo/291640
|
||||
2023-08-01 02:24:00:【photo.before】开始下载章节: 291640 (145504[104/104]), 标题: [健身教练 完結], 图片数为[41]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00001.webp [1/41], [https://cdn-msp.18comic.vip/media/photos/291640/00001.webp] → [D:/jmcomic/145504/00001.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00004.webp [4/41], [https://cdn-msp.18comic.vip/media/photos/291640/00004.webp] → [D:/jmcomic/145504/00004.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00003.webp [3/41], [https://cdn-msp.18comic.vip/media/photos/291640/00003.webp] → [D:/jmcomic/145504/00003.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00007.webp [7/41], [https://cdn-msp.18comic.vip/media/photos/291640/00007.webp] → [D:/jmcomic/145504/00007.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00005.webp [5/41], [https://cdn-msp.18comic.vip/media/photos/291640/00005.webp] → [D:/jmcomic/145504/00005.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00002.webp [2/41], [https://cdn-msp.18comic.vip/media/photos/291640/00002.webp] → [D:/jmcomic/145504/00002.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00008.webp [8/41], [https://cdn-msp.18comic.vip/media/photos/291640/00008.webp] → [D:/jmcomic/145504/00008.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00006.webp [6/41], [https://cdn-msp.18comic.vip/media/photos/291640/00006.webp] → [D:/jmcomic/145504/00006.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00009.webp [9/41], [https://cdn-msp.18comic.vip/media/photos/291640/00009.webp] → [D:/jmcomic/145504/00009.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00010.webp [10/41], [https://cdn-msp.18comic.vip/media/photos/291640/00010.webp] → [D:/jmcomic/145504/00010.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00011.webp [11/41], [https://cdn-msp.18comic.vip/media/photos/291640/00011.webp] → [D:/jmcomic/145504/00011.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00012.webp [12/41], [https://cdn-msp.18comic.vip/media/photos/291640/00012.webp] → [D:/jmcomic/145504/00012.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00013.webp [13/41], [https://cdn-msp.18comic.vip/media/photos/291640/00013.webp] → [D:/jmcomic/145504/00013.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00014.webp [14/41], [https://cdn-msp.18comic.vip/media/photos/291640/00014.webp] → [D:/jmcomic/145504/00014.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00015.webp [15/41], [https://cdn-msp.18comic.vip/media/photos/291640/00015.webp] → [D:/jmcomic/145504/00015.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00016.webp [16/41], [https://cdn-msp.18comic.vip/media/photos/291640/00016.webp] → [D:/jmcomic/145504/00016.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00017.webp [17/41], [https://cdn-msp.18comic.vip/media/photos/291640/00017.webp] → [D:/jmcomic/145504/00017.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00018.webp [18/41], [https://cdn-msp.18comic.vip/media/photos/291640/00018.webp] → [D:/jmcomic/145504/00018.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00019.webp [19/41], [https://cdn-msp.18comic.vip/media/photos/291640/00019.webp] → [D:/jmcomic/145504/00019.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00020.webp [20/41], [https://cdn-msp.18comic.vip/media/photos/291640/00020.webp] → [D:/jmcomic/145504/00020.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00021.webp [21/41], [https://cdn-msp.18comic.vip/media/photos/291640/00021.webp] → [D:/jmcomic/145504/00021.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00023.webp [23/41], [https://cdn-msp.18comic.vip/media/photos/291640/00023.webp] → [D:/jmcomic/145504/00023.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00022.webp [22/41], [https://cdn-msp.18comic.vip/media/photos/291640/00022.webp] → [D:/jmcomic/145504/00022.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00024.webp [24/41], [https://cdn-msp.18comic.vip/media/photos/291640/00024.webp] → [D:/jmcomic/145504/00024.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00025.webp [25/41], [https://cdn-msp.18comic.vip/media/photos/291640/00025.webp] → [D:/jmcomic/145504/00025.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00027.webp [27/41], [https://cdn-msp.18comic.vip/media/photos/291640/00027.webp] → [D:/jmcomic/145504/00027.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00026.webp [26/41], [https://cdn-msp.18comic.vip/media/photos/291640/00026.webp] → [D:/jmcomic/145504/00026.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00028.webp [28/41], [https://cdn-msp.18comic.vip/media/photos/291640/00028.webp] → [D:/jmcomic/145504/00028.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00029.webp [29/41], [https://cdn-msp.18comic.vip/media/photos/291640/00029.webp] → [D:/jmcomic/145504/00029.jpg]
|
||||
2023-08-01 02:24:00:【image.before】图片准备下载: 291640/00030.webp [30/41], [https://cdn-msp.18comic.vip/media/photos/291640/00030.webp] → [D:/jmcomic/145504/00030.jpg]
|
||||
2023-08-01 02:24:01:【image.after】图片下载完成: 291640/00017.webp [17/41], [https://cdn-msp.18comic.vip/media/photos/291640/00017.webp] → [D:/jmcomic/145504/00017.jpg]
|
||||
2023-08-01 02:24:01:【image.before】图片准备下载: 291640/00031.webp [31/41], [https://cdn-msp.18comic.vip/media/photos/291640/00031.webp] → [D:/jmcomic/145504/00031.jpg]
|
||||
2023-08-01 02:24:01:【image.after】图片下载完成: 291640/00007.webp [7/41], [https://cdn-msp.18comic.vip/media/photos/291640/00007.webp] → [D:/jmcomic/145504/00007.jpg]
|
||||
2023-08-01 02:24:01:【image.before】图片准备下载: 291640/00032.webp [32/41], [https://cdn-msp.18comic.vip/media/photos/291640/00032.webp] → [D:/jmcomic/145504/00032.jpg]
|
||||
2023-08-01 02:24:01:【image.after】图片下载完成: 291640/00021.webp [21/41], [https://cdn-msp.18comic.vip/media/photos/291640/00021.webp] → [D:/jmcomic/145504/00021.jpg]
|
||||
2023-08-01 02:24:01:【image.before】图片准备下载: 291640/00033.webp [33/41], [https://cdn-msp.18comic.vip/media/photos/291640/00033.webp] → [D:/jmcomic/145504/00033.jpg]
|
||||
2023-08-01 02:24:02:【image.after】图片下载完成: 291640/00003.webp [3/41], [https://cdn-msp.18comic.vip/media/photos/291640/00003.webp] → [D:/jmcomic/145504/00003.jpg]
|
||||
2023-08-01 02:24:02:【image.before】图片准备下载: 291640/00034.webp [34/41], [https://cdn-msp.18comic.vip/media/photos/291640/00034.webp] → [D:/jmcomic/145504/00034.jpg]
|
||||
2023-08-01 02:24:03:【image.after】图片下载完成: 291640/00008.webp [8/41], [https://cdn-msp.18comic.vip/media/photos/291640/00008.webp] → [D:/jmcomic/145504/00008.jpg]
|
||||
2023-08-01 02:24:03:【image.before】图片准备下载: 291640/00035.webp [35/41], [https://cdn-msp.18comic.vip/media/photos/291640/00035.webp] → [D:/jmcomic/145504/00035.jpg]
|
||||
2023-08-01 02:24:04:【image.after】图片下载完成: 291640/00015.webp [15/41], [https://cdn-msp.18comic.vip/media/photos/291640/00015.webp] → [D:/jmcomic/145504/00015.jpg]
|
||||
2023-08-01 02:24:04:【image.before】图片准备下载: 291640/00036.webp [36/41], [https://cdn-msp.18comic.vip/media/photos/291640/00036.webp] → [D:/jmcomic/145504/00036.jpg]
|
||||
2023-08-01 02:24:04:【image.after】图片下载完成: 291640/00029.webp [29/41], [https://cdn-msp.18comic.vip/media/photos/291640/00029.webp] → [D:/jmcomic/145504/00029.jpg]
|
||||
2023-08-01 02:24:04:【image.after】图片下载完成: 291640/00030.webp [30/41], [https://cdn-msp.18comic.vip/media/photos/291640/00030.webp] → [D:/jmcomic/145504/00030.jpg]
|
||||
2023-08-01 02:24:04:【image.before】图片准备下载: 291640/00037.webp [37/41], [https://cdn-msp.18comic.vip/media/photos/291640/00037.webp] → [D:/jmcomic/145504/00037.jpg]
|
||||
2023-08-01 02:24:04:【image.before】图片准备下载: 291640/00038.webp [38/41], [https://cdn-msp.18comic.vip/media/photos/291640/00038.webp] → [D:/jmcomic/145504/00038.jpg]
|
||||
2023-08-01 02:24:04:【image.after】图片下载完成: 291640/00019.webp [19/41], [https://cdn-msp.18comic.vip/media/photos/291640/00019.webp] → [D:/jmcomic/145504/00019.jpg]
|
||||
2023-08-01 02:24:04:【image.before】图片准备下载: 291640/00039.webp [39/41], [https://cdn-msp.18comic.vip/media/photos/291640/00039.webp] → [D:/jmcomic/145504/00039.jpg]
|
||||
2023-08-01 02:24:04:【image.after】图片下载完成: 291640/00024.webp [24/41], [https://cdn-msp.18comic.vip/media/photos/291640/00024.webp] → [D:/jmcomic/145504/00024.jpg]
|
||||
2023-08-01 02:24:04:【image.before】图片准备下载: 291640/00040.webp [40/41], [https://cdn-msp.18comic.vip/media/photos/291640/00040.webp] → [D:/jmcomic/145504/00040.jpg]
|
||||
2023-08-01 02:24:04:【image.after】图片下载完成: 291640/00026.webp [26/41], [https://cdn-msp.18comic.vip/media/photos/291640/00026.webp] → [D:/jmcomic/145504/00026.jpg]
|
||||
2023-08-01 02:24:04:【image.before】图片准备下载: 291640/00041.webp [41/41], [https://cdn-msp.18comic.vip/media/photos/291640/00041.webp] → [D:/jmcomic/145504/00041.jpg]
|
||||
2023-08-01 02:24:04:【image.after】图片下载完成: 291640/00001.webp [1/41], [https://cdn-msp.18comic.vip/media/photos/291640/00001.webp] → [D:/jmcomic/145504/00001.jpg]
|
||||
2023-08-01 02:24:04:【image.after】图片下载完成: 291640/00010.webp [10/41], [https://cdn-msp.18comic.vip/media/photos/291640/00010.webp] → [D:/jmcomic/145504/00010.jpg]
|
||||
2023-08-01 02:24:04:【image.after】图片下载完成: 291640/00012.webp [12/41], [https://cdn-msp.18comic.vip/media/photos/291640/00012.webp] → [D:/jmcomic/145504/00012.jpg]
|
||||
2023-08-01 02:24:04:【image.after】图片下载完成: 291640/00032.webp [32/41], [https://cdn-msp.18comic.vip/media/photos/291640/00032.webp] → [D:/jmcomic/145504/00032.jpg]
|
||||
2023-08-01 02:24:04:【image.after】图片下载完成: 291640/00011.webp [11/41], [https://cdn-msp.18comic.vip/media/photos/291640/00011.webp] → [D:/jmcomic/145504/00011.jpg]
|
||||
2023-08-01 02:24:04:【image.after】图片下载完成: 291640/00034.webp [34/41], [https://cdn-msp.18comic.vip/media/photos/291640/00034.webp] → [D:/jmcomic/145504/00034.jpg]
|
||||
2023-08-01 02:24:05:【image.after】图片下载完成: 291640/00023.webp [23/41], [https://cdn-msp.18comic.vip/media/photos/291640/00023.webp] → [D:/jmcomic/145504/00023.jpg]
|
||||
2023-08-01 02:24:05:【image.after】图片下载完成: 291640/00002.webp [2/41], [https://cdn-msp.18comic.vip/media/photos/291640/00002.webp] → [D:/jmcomic/145504/00002.jpg]
|
||||
2023-08-01 02:24:05:【image.after】图片下载完成: 291640/00038.webp [38/41], [https://cdn-msp.18comic.vip/media/photos/291640/00038.webp] → [D:/jmcomic/145504/00038.jpg]
|
||||
2023-08-01 02:24:05:【image.after】图片下载完成: 291640/00041.webp [41/41], [https://cdn-msp.18comic.vip/media/photos/291640/00041.webp] → [D:/jmcomic/145504/00041.jpg]
|
||||
2023-08-01 02:24:06:【image.after】图片下载完成: 291640/00035.webp [35/41], [https://cdn-msp.18comic.vip/media/photos/291640/00035.webp] → [D:/jmcomic/145504/00035.jpg]
|
||||
2023-08-01 02:24:06:【image.after】图片下载完成: 291640/00009.webp [9/41], [https://cdn-msp.18comic.vip/media/photos/291640/00009.webp] → [D:/jmcomic/145504/00009.jpg]
|
||||
2023-08-01 02:24:06:【image.after】图片下载完成: 291640/00031.webp [31/41], [https://cdn-msp.18comic.vip/media/photos/291640/00031.webp] → [D:/jmcomic/145504/00031.jpg]
|
||||
2023-08-01 02:24:06:【image.after】图片下载完成: 291640/00033.webp [33/41], [https://cdn-msp.18comic.vip/media/photos/291640/00033.webp] → [D:/jmcomic/145504/00033.jpg]
|
||||
2023-08-01 02:24:06:【image.after】图片下载完成: 291640/00027.webp [27/41], [https://cdn-msp.18comic.vip/media/photos/291640/00027.webp] → [D:/jmcomic/145504/00027.jpg]
|
||||
2023-08-01 02:24:06:【image.after】图片下载完成: 291640/00039.webp [39/41], [https://cdn-msp.18comic.vip/media/photos/291640/00039.webp] → [D:/jmcomic/145504/00039.jpg]
|
||||
2023-08-01 02:24:06:【image.after】图片下载完成: 291640/00036.webp [36/41], [https://cdn-msp.18comic.vip/media/photos/291640/00036.webp] → [D:/jmcomic/145504/00036.jpg]
|
||||
2023-08-01 02:24:06:【image.after】图片下载完成: 291640/00020.webp [20/41], [https://cdn-msp.18comic.vip/media/photos/291640/00020.webp] → [D:/jmcomic/145504/00020.jpg]
|
||||
2023-08-01 02:24:06:【image.after】图片下载完成: 291640/00037.webp [37/41], [https://cdn-msp.18comic.vip/media/photos/291640/00037.webp] → [D:/jmcomic/145504/00037.jpg]
|
||||
2023-08-01 02:24:06:【image.after】图片下载完成: 291640/00040.webp [40/41], [https://cdn-msp.18comic.vip/media/photos/291640/00040.webp] → [D:/jmcomic/145504/00040.jpg]
|
||||
2023-08-01 02:24:06:【image.after】图片下载完成: 291640/00004.webp [4/41], [https://cdn-msp.18comic.vip/media/photos/291640/00004.webp] → [D:/jmcomic/145504/00004.jpg]
|
||||
2023-08-01 02:24:07:【image.after】图片下载完成: 291640/00005.webp [5/41], [https://cdn-msp.18comic.vip/media/photos/291640/00005.webp] → [D:/jmcomic/145504/00005.jpg]
|
||||
2023-08-01 02:24:07:【image.after】图片下载完成: 291640/00025.webp [25/41], [https://cdn-msp.18comic.vip/media/photos/291640/00025.webp] → [D:/jmcomic/145504/00025.jpg]
|
||||
2023-08-01 02:24:07:【image.after】图片下载完成: 291640/00014.webp [14/41], [https://cdn-msp.18comic.vip/media/photos/291640/00014.webp] → [D:/jmcomic/145504/00014.jpg]
|
||||
2023-08-01 02:24:07:【image.after】图片下载完成: 291640/00028.webp [28/41], [https://cdn-msp.18comic.vip/media/photos/291640/00028.webp] → [D:/jmcomic/145504/00028.jpg]
|
||||
2023-08-01 02:24:07:【image.after】图片下载完成: 291640/00018.webp [18/41], [https://cdn-msp.18comic.vip/media/photos/291640/00018.webp] → [D:/jmcomic/145504/00018.jpg]
|
||||
2023-08-01 02:24:08:【image.after】图片下载完成: 291640/00022.webp [22/41], [https://cdn-msp.18comic.vip/media/photos/291640/00022.webp] → [D:/jmcomic/145504/00022.jpg]
|
||||
2023-08-01 02:24:08:【image.after】图片下载完成: 291640/00016.webp [16/41], [https://cdn-msp.18comic.vip/media/photos/291640/00016.webp] → [D:/jmcomic/145504/00016.jpg]
|
||||
2023-08-01 02:24:08:【image.after】图片下载完成: 291640/00006.webp [6/41], [https://cdn-msp.18comic.vip/media/photos/291640/00006.webp] → [D:/jmcomic/145504/00006.jpg]
|
||||
2023-08-01 02:24:08:【image.after】图片下载完成: 291640/00013.webp [13/41], [https://cdn-msp.18comic.vip/media/photos/291640/00013.webp] → [D:/jmcomic/145504/00013.jpg]
|
||||
2023-08-01 02:24:08:【photo.after】章节下载完成: [291640] (145504[104/104])
|
||||
2023-08-01 02:24:08:【album.after】本子下载完成: [145504]
|
||||
2023-08-01 02:24:08:【plugin.invoke】调用插件: [zip]
|
||||
2023-08-01 02:24:08:【plugin.zip.finish】压缩章节[291640]成功 → D:/jmcomic/健身教练 完結.zip
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00017.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00007.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00021.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00003.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00008.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00015.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00029.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00030.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00019.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00024.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00026.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00001.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00010.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00012.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00032.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00011.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00034.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00023.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00002.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00038.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00041.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00035.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00009.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00031.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00033.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00027.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00039.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00036.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00020.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00037.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00040.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00004.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00005.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00025.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00014.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00028.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00018.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00022.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00016.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00006.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除原文件: D:/jmcomic/145504/00013.jpg
|
||||
2023-08-01 02:24:08:【plugin.zip.remove】移除文件夹: D:/jmcomic/145504
|
||||
|
||||
进程已结束,退出代码为 0
|
||||
|
||||
"""
|
@ -1,27 +0,0 @@
|
||||
"""
|
||||
|
||||
使用命令行下载禁漫本子
|
||||
|
||||
1. 基本用法
|
||||
# 下载album 123 456,下载photo 333。彼此之间使用空格间隔
|
||||
```
|
||||
jmcomic 123 456 p333
|
||||
```
|
||||
|
||||
2. 自定义option
|
||||
|
||||
2.1. 通过命令行
|
||||
```
|
||||
jmcomic 123 --option="D:/a.yml"
|
||||
```
|
||||
|
||||
2.2. 通过环境变量
|
||||
设置环境: JM_OPTION_PATH = D:/a.yml
|
||||
命令行的命令不变
|
||||
```
|
||||
jmcomic 123
|
||||
```
|
||||
|
||||
|
||||
|
||||
"""
|
@ -1,41 +0,0 @@
|
||||
"""
|
||||
|
||||
该文件介绍不同客户端、以及配置客户端实现的方式(移动端、网页端)
|
||||
|
||||
有关不同客户端的介绍:
|
||||
|
||||
网页端:
|
||||
代号为 html
|
||||
最早实现的客户端,由于网页信息较为丰富,相对而言效率比较高
|
||||
但是不同域名会限制ip,比如美国ip不能使用 18comic.vip 域名
|
||||
不同域名的网站稳定性也差别很大
|
||||
国内直连域名经常换
|
||||
|
||||
移动端:
|
||||
代号为 api
|
||||
在v2.3.3版本中引入的新客户端实现
|
||||
比网页端效率比较低,因为同样的信息量,移动端需要请求多个接口
|
||||
但是不限制ip地区,兼容性很好,国内直连比较稳定
|
||||
|
||||
|
||||
|
||||
1. 通过配置文件
|
||||
```yml
|
||||
client:
|
||||
# 下面两行只能选一行放到你的配置文件
|
||||
impl: api # 移动端
|
||||
impl: html # 网页端
|
||||
|
||||
```
|
||||
|
||||
2. 通过修改模块配置类 JmModuleConfig
|
||||
|
||||
# JmModuleConfig.CLIENT_IMPL_DEFAULT 该字段表示默认的客户端实现
|
||||
# 当配置文件中不存在client.impl配置时,就使用 JmModuleConfig.CLIENT_IMPL_DEFAULT
|
||||
# 你可以不在配置文件中配置,而是使用下面的代码(在你创建option前)
|
||||
|
||||
JmModuleConfig.CLIENT_IMPL_DEFAULT = 'api' # 移动端
|
||||
JmModuleConfig.CLIENT_IMPL_DEFAULT = 'html' # 网页端
|
||||
|
||||
|
||||
"""
|
@ -1,91 +0,0 @@
|
||||
"""
|
||||
|
||||
本文件仅演示一些简单的api使用,包含以下内容:
|
||||
1. 下载本子
|
||||
2. 获取实体类(本子/章节/图片)
|
||||
3. 搜索本子
|
||||
4. 搜索并下载本子(以下载带有 [無修正] 标签的本子为例)
|
||||
|
||||
"""
|
||||
|
||||
from jmcomic import *
|
||||
|
||||
# 核心下载配置
|
||||
option = create_option(
|
||||
f'你的配置文件路径,例如: D:/a/b/c/jmcomic/config.yml'
|
||||
)
|
||||
# 提供请求功能的客户端对象
|
||||
client = option.build_jm_client()
|
||||
|
||||
|
||||
@timeit('下载本子: ')
|
||||
def download_jm_album():
|
||||
ls = str_to_list('''
|
||||
438696
|
||||
https://18comic.vip/album/497896/
|
||||
|
||||
''')
|
||||
|
||||
download_album(ls, option)
|
||||
|
||||
|
||||
@timeit('获取实体类: ')
|
||||
def get_album_photo_detail():
|
||||
# 本子实体类
|
||||
album: JmAlbumDetail = client.get_album_detail('427413')
|
||||
|
||||
def show(photo: JmPhotoDetail):
|
||||
# 章节实体类
|
||||
photo = client.get_photo_detail(photo.photo_id, False)
|
||||
|
||||
# 图片实体类
|
||||
image: JmImageDetail
|
||||
for image in photo:
|
||||
print(image.img_url)
|
||||
|
||||
multi_thread_launcher(
|
||||
iter_objs=album,
|
||||
apply_each_obj_func=show
|
||||
)
|
||||
|
||||
|
||||
@timeit('搜索本子: ')
|
||||
def search_jm_album():
|
||||
# 分页查询,search_site就是禁漫网页上的【站内搜索】
|
||||
page: JmSearchPage = client.search_site(search_query='+MANA +无修正', page=1)
|
||||
# page默认的迭代方式是page.iter_id_title(),每次迭代返回 albun_id, title
|
||||
for album_id, title in page:
|
||||
print(f'[{album_id}]: {title}')
|
||||
|
||||
# 直接搜索禁漫车号
|
||||
page = client.search_site(search_query='427413')
|
||||
album: JmAlbumDetail = page.single_album
|
||||
print(album.keywords)
|
||||
|
||||
|
||||
@timeit('搜索并下载本子: ')
|
||||
def search_and_download():
|
||||
tag = '無修正'
|
||||
# 搜索标签,可以使用search_tag。
|
||||
# 搜索第一页。
|
||||
page: JmSearchPage = client.search_tag(tag, page=1)
|
||||
|
||||
aid_list = []
|
||||
|
||||
for aid, atitle, tag_list in page.iter_id_title_tag(): # 使用page的iter_id_title_tag迭代器
|
||||
if tag in tag_list:
|
||||
print(f'[标签/{tag}] 发现目标: [{aid}]: [{atitle}]')
|
||||
aid_list.append(aid)
|
||||
|
||||
download_album(aid_list, option)
|
||||
|
||||
|
||||
def main():
|
||||
search_jm_album()
|
||||
download_jm_album()
|
||||
get_album_photo_detail()
|
||||
search_and_download()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
@ -44,7 +44,7 @@ def main():
|
||||
|
||||
def get_option():
|
||||
# 读取 option 配置文件
|
||||
option = create_option('../assets/config/option_workflow_download.yml')
|
||||
option = create_option('../assets/option/option_workflow_download.yml')
|
||||
|
||||
# 支持工作流覆盖配置文件的配置
|
||||
cover_option_config(option)
|
||||
|