Py-3rdparty-mediawiki
What is it
Extended functionality for
- pywikibot
- mwclient
Github
Installation
git clone https://github.com/WolfgangFahl/py-3rdparty-mediawiki
./install
wikipush / wikiedit / wiknuke / wikiuser scripts
This script is in the scripts directory. It should be linked with three names
ln wikipush wikiuser
ln wikipush wikinuke
ln wikipush wikiedit
and be added to your path e.g by doing a symbolic link
cd $HOME/bin
ln -s $HOME/Documents/pyworkspace/py-3rdparty-mediawiki/scripts/wikipush .
ln -s $HOME/Documents/pyworkspace/py-3rdparty-mediawiki/scripts/wikiuser .
ln -s $HOME/Documents/pyworkspace/py-3rdparty-mediawiki/scripts/wikinuke .
ln -s $HOME/Documents/pyworkspace/py-3rdparty-mediawiki/scripts/wikiedit .
#!/bin/bash
# WF 2020-10-31
# wrapper for wikipush python
script=$(python -c "import os;import sys;print (os.path.realpath(sys.argv[1]))" $BASH_SOURCE)
scriptname=$(basename $script)
scriptdir=$(dirname $script)
base=$scriptdir/..
export PYTHONPATH="${PYTHONPATH}:$base"
case $scriptname in
"wikipush")
python -m wikibot.wikipush "$@"
;;
"wikiedit")
python -m wikibot.wikiedit "$@"
;;
"wikiuser")
python -m wikibot.wikiuser "$@"
;;
"wikinuke")
python -m wikibot.wikinuke "$@"
;;
*)
echo "undefined wikipush script behavior: $scriptname"
;;
esac
WikiPush
WikiPush allows to copy pages from one wiki to another including the images on the page. To identify yourself you use the credential property files created with the wikiuser script (using python) or the Mediawiki-Japi CommandLine
usage
wikipush -h
family and mylang are not set.
Defaulting to family='test' and mylang='test'.
usage: wikipush.py [-h] [-d] [-V] [-l] [-f] [-i] [-q QUERY] -s SOURCE -t
TARGET [-p PAGES [PAGES ...]]
Created on 2020-10-29
Created by Wolfgang Fahl on 2020-10-31.
Copyright 2020 Wolfgang Fahl. All rights reserved.
Licensed under the Apache License 2.0
http://www.apache.org/licenses/LICENSE-2.0
Distributed on an "AS IS" basis without warranties
or conditions of any kind, either express or implied.
USAGE
optional arguments:
-h, --help show this help message and exit
-d, --debug set debug level [default: None]
-V, --version show program's version number and exit
-l, --login login to source wiki for access permission
-f, --force force to overwrite existing pages
-i, --ignore ignore upload warnings e.g. duplicate images
-q QUERY, --query QUERY
select pages with given SMW ask query
-s SOURCE, --source SOURCE
source wiki id
-t TARGET, --target TARGET
target wiki id
-p PAGES [PAGES ...], --pages PAGES [PAGES ...]
list of page Titles to be pushed
Example
wikipush -s smw -t test2 -q "[[Category:City]]|limit=5"
family and mylang are not set.
Defaulting to family='test' and mylang='test'.
copying 4 pages from smw to test2
copying Demo:Tokyo ...✅
copying image File:SMW-Info-button.png ...✅
copying image File:Tokyo-Tsukishima-0011.jpg ...✅
copying Vienna ...✅
copying Warsaw ...✅
copying image File:6140285934 02e81b845f z.jpg ...✅
copying Demo:Würzburg ...✅
WikiNuke
wikinukes.py allows mass deletion of pages
usage
usage: wikinuke.py [-h] [-d] [-V] [-f] [-q QUERY] [-qf QUERYFIELD] -t TARGET [-p PAGES [PAGES ...]]
Created on 2020-11-12
Created by Wolfgang Fahl on 2020-10-31.
Copyright 2020 Wolfgang Fahl. All rights reserved.
Licensed under the Apache License 2.0
http://www.apache.org/licenses/LICENSE-2.0
Distributed on an "AS IS" basis without warranties
or conditions of any kind, either express or implied.
USAGE
optional arguments:
-h, --help show this help message and exit
-d, --debug set debug level [default: None]
-V, --version show program's version number and exit
-f, --force force to delete pages - default is 'dry' run only listing pages
-q QUERY, --query QUERY
select pages with given SMW ask query
-qf QUERYFIELD, --queryField QUERYFIELD
query result field which contains page
-t TARGET, --target TARGET
target wiki id
-p PAGES [PAGES ...], --pages PAGES [PAGES ...]
list of page Titles to be pushed
Example
The default behavior is a dry run only listing whether the pages exist
deleting 3 pages in test (dry run)
1/3 ( 33%): deleting deleteMe1 ...👍
2/3 ( 67%): deleting deleteMe2 ...👍
3/3 ( 100%): deleting deleteMe3 ...👍
After checking you might want to (carefully) use the "-f" option to actually force the deletion:
wikinuke -t test -p deleteMe1 deleteMe2 deleteMe3 -f
deleting 3 pages in test (forced)
1/3 ( 33%): deleting deleteMe1 ...✅
2/3 ( 67%): deleting deleteMe2 ...✅
3/3 ( 100%): deleting deleteMe3 ...✅
WikiEdit
wikiedit.py allows mass editing of pages using python regular expressions
usage
wikiedit -h
usage: wikiedit.py [-h] [-d] [-V] --search SEARCH --replace REPLACE [-f] [-q QUERY] [-qf QUERYFIELD] -t TARGET
[-p PAGES [PAGES ...]]
Created on 2020-11-12
Created by Wolfgang Fahl on 2020-10-31.
Copyright 2020 Wolfgang Fahl. All rights reserved.
Licensed under the Apache License 2.0
http://www.apache.org/licenses/LICENSE-2.0
Distributed on an "AS IS" basis without warranties
or conditions of any kind, either express or implied.
USAGE
optional arguments:
-h, --help show this help message and exit
-d, --debug set debug level [default: None]
-V, --version show program's version number and exit
--search SEARCH search pattern
--replace REPLACE replace pattern
-f, --force force to edit pages - default is 'dry' run only listing pages
-q QUERY, --query QUERY
select pages with given SMW ask query
-qf QUERYFIELD, --queryField QUERYFIELD
query result field which contains page
-t TARGET, --target TARGET
target wiki id
-p PAGES [PAGES ...], --pages PAGES [PAGES ...]
list of page Titles to be pushed
WikiUser
wikiuser.py creates credential files and assigns a WikiId under which you can now operate. This simplifies access to your wiki. The credential file is compatible to the Java Mediawiki-Japi see CommandLine#Credential_mode
usage
wikiuser -h
usage: wikiuser.py [-h] [-d] [-V] [-e EMAIL] [-f FILEPATH] [-l URL]
[-s SCRIPTPATH] [-p PASSWORD] [-u USER] [-v VERSION]
[-w WIKIID] [-y]
WikiUser credential handling
Created by Wolfgang Fahl on 2020-10-31.
Copyright 2020 Wolfgang Fahl. All rights reserved.
Licensed under the Apache License 2.0
http://www.apache.org/licenses/LICENSE-2.0
Distributed on an "AS IS" basis without warranties
or conditions of any kind, either express or implied.
USAGE
optional arguments:
-h, --help show this help message and exit
-d, --debug set debug level [default: None]
-V, --version show program's version number and exit
-e EMAIL, --email EMAIL
email of the user
-f FILEPATH, --file FILEPATH
ini-file path
-l URL, --url URL url of the wiki
-s SCRIPTPATH, --scriptPath SCRIPTPATH
script path
-p PASSWORD, --password PASSWORD
password
-u USER, --user USER os user id
-v VERSION, --wikiVersion VERSION
version of the wiki
-w WIKIID, --wikiId WIKIID
wiki Id
-y, --yes immediately store without asking
Example
E.g. if you have an account on www.semantic-mediawiki.org you can start wikiuser in interactive mode.
wikiuser
email: john@doe.com
scriptPath: /w
user: jd
url: http://www.semantic-mediawiki.org
version: Mediawiki 1.33
wikiId: smw
password: *****
shall i store jd smw? yes/no y/ny
Now you can e.g. use "smw" as the wikiid for this wiki when using wikipush
Prerequisites
You might want to prepare some credential ini files with the wikiuser script or Mediawiki-Japi CommandLine.
user-config.py
pywikibot expects a user-config.py file. The minimum recommended file for intranet usecases is:
# https://stackoverflow.com/a/60885381/1497139
# Slow down the robot such that it never makes a second page edit within
# 'put_throttle' seconds.
put_throttle = 0
# avoid warnings ...
family='bitplan'
mylang='en'
The easiest way is to put it at $HOME/.pywikibot/user-config.py
Features
Encrypted credential handling
Py-3rdparty-mediawiki allows using pywikibot by simply giving each wiki an id and using the credential information created by MediaWiki-Japi. The needed family file is automatically created and registered. If you'd like to get a pure python solution for credential handling please file an issue on github - it's no big deal but i personally don't need it yet since i'm fine with the new CommandLine feature added recently.
Semantic MediaWiki API support
see https://github.com/WolfgangFahl/py-3rdparty-mediawiki/issues/1
Example
from wikibot.wikibot import WikiBot
wikibot=WikiBot.ofWikiId("test2")
wikibot.site ...