Skip to content

Commit

Permalink
chore: add custom configuration demo
Browse files Browse the repository at this point in the history
  • Loading branch information
shengchenyang committed Nov 14, 2024
1 parent d97709b commit 09a9f49
Show file tree
Hide file tree
Showing 2 changed files with 51 additions and 0 deletions.
21 changes: 21 additions & 0 deletions ayugespidertools/templates/project/module/VIT/.conf
Original file line number Diff line number Diff line change
Expand Up @@ -192,3 +192,24 @@
;upload_fields_suffix=_file_url
;oss_fields_prefix=_
;full_link_enable=false

; Custom configuration:
; you can create a custom section here to securely store private or project-specific
; settings. This configuration can be combined with the `get_cfg` parsing method
; to load custom values. for example:
; ```
; from ayugespidertools.config import get_cfg
;
; _my_cfg = get_cfg()
; custom_option = _my_cfg["custom_section"].get("custom_option", "no_custom_value")
; custom_int = _my_cfg["custom_section"].getint("custom_int", 0)
; custom_bool = _my_cfg["custom_section"].getboolean("custom_bool", False)
; custom_float = _my_cfg["custom_section"].getfloat("custom_float", 3.14)
; ```
; See `topics/configuration` for a complete example.

;[custom_section]
;custom_option=custom_value
;custom_int=1
;custom_bool=true
;custom_float=3.1415926
30 changes: 30 additions & 0 deletions docs/topics/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,3 +191,33 @@
| full_link_enable | 是否开启完整链接,默认 false | 为是否保存完整的 oss 文件链接。 |

遵守规则时的 `oss` 上传逻辑时使用,更复杂的需求也可根据示例自行实现。具体请看 `demo_oss``demo_oss_sec` 的场景示例。请自行选择可接受的风格。

## [custom_section]

用于自定义配置:

一些 `scrapy` 第三方扩展需要在 `settings.py` 中设置一些配置,涉及到 `host`,密码等隐私配置,直接展示在 `settings.py` 里是不可接受的,这里提供一种方法来解决。

`settings.py``spider` 等脚本中赋值重要参数时,可以从 `VIT_DIR``.conf` 中获取自定义配置内容,来达到隐藏隐私内容和保持配置内容统一存放的目的;

比如在 `.conf` 中自定义配置以下内容:

```shell
[custom_section]
custom_option=custom_value
custom_int=1
custom_bool=true
custom_float=3.1415926
```

那么,可以在程序任意地方通过 `get_cfg` 来获取自定义部分:

```python
from ayugespidertools.config import get_cfg

_my_cfg = get_cfg()
custom_option = _my_cfg["custom_section"].get("custom_option", "no_custom_value")
custom_int = _my_cfg["custom_section"].getint("custom_int", 0)
custom_bool = _my_cfg["custom_section"].getboolean("custom_bool", False)
custom_float = _my_cfg["custom_section"].getfloat("custom_float", 3.14)
```

0 comments on commit 09a9f49

Please sign in to comment.