0

I recently decided to save the english wikibooks for offline access, and I went with setting up mediawiki, importing a xml dump, and configuring it to work. I had done the first two, following instructions on mediawiki.org, but I'm getting errors.

I have done the following so far:

  • Installed the newest version of php using homebrew
  • Downloading and copying the newest version of mediawiki (1.35) to /Library/WebServer/Documents
  • Went through the setup and generated LocalSettings.php (content at the end of the post)
  • Imported the dump using cd /Library/WebServer/Documents/mediawiki/maintenance/ ; php importDump.php --conf ../LocalSettings.php /volumes/data/tmp/enwikibooks-20201201-pages-meta-current.xml --username-prefix=""
    • Installed TemplateStyles because it is needed for the import: wfLoadExtension( 'TemplateStyles' );
  • (I can browse the wiki now, but math rendering is completely disabled)
  • Installed the math extension:
    • Extracted the newest version at /Library/WebServer/Documents/mediawiki/extensions/
    • Added the following to LocalSettings.php:
      wfLoadExtension( 'Math' );
      $wgMathValidModes[] = 'mathml';
      $wgDefaultUserOptions['math'] = 'mathml';
      $wgMathMathMLUrl = 'http://localhost:10042/';
      
    • Installed Mathoid and Restbase
      • npm -g config set user root
      • npm install -g mathoid
      • npm install -g restbase
      • mathoid runs without problem (passes the test on mediawiki)
      • restbase didn't. I just get {"type":"https://mediawiki.org/wiki/HyperSwitch/errors/not_found#route","title":"Not found.","method":"get","uri":"/localhost/v1/page/html/index.php/Main_Page"} when I execute curl http://localhost:7231/localhost/v1/page/html/index.php/Main_Page. However, I can access http://localhost:7231/localhost/v1 without any problems. (After I changed the configuration further, it gives me an error: enter image description here
  • I ran necessary tools to update the database (cd /Library/WebServer/Documents/mediawiki/maintenance/):
    • php initSiteStats.php --update
    • php rebuildall.php (aborted at "refreshing links tables" stage because it's way too slow and give me error Notice: Trying to get property 'identifiers' of non-object in /Library/WebServer/Documents/mediawiki/extensions/Math/src/MathRestbaseInterface.php on line 396 which might be related to the need to run update.php after installing the math extension)
    • php update.php
  • After setting up the math extension and necessary tools it need, for whatever I put for "$wgMathFullRestbaseURL", it is giving me errors. Example:
Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \lim_{x\to2}f(x)=4}

This message is due to GFW blocking wikipedia-related websites, because some math is converted into svg. However, when I change it to the local restbase url, which I can't find any documentation of (so I tried a bunch of things that looked similar), all of math is not rendered and I have the same error message (with different URLs) all over the place.

What URL should I put?

LocalSettings.php:

<?php
# This file was automatically generated by the MediaWiki 1.35.1
# installer. If you make manual changes, please keep track in case you
# need to recreate them later.
#
# See includes/DefaultSettings.php for all configurable settings
# and their default values, but don't forget to make changes in _this_
# file, not there.
#
# Further documentation for configuration settings may be found at:
# https://www.mediawiki.org/wiki/Manual:Configuration_settings

# Protect against web entry
if ( !defined( 'MEDIAWIKI' ) ) {
    exit;
}


## Uncomment this to disable output compression
# $wgDisableOutputCompression = true;

$wgSitename = "enwikibooks";
$wgMetaNamespace = "Enwikibooks";

## The URL base path to the directory containing the wiki;
## defaults for all runtime URL paths are based off of this.
## For more information on customizing the URLs
## (like /w/index.php/Page_title to /wiki/Page_title) please see:
## https://www.mediawiki.org/wiki/Manual:Short_URL
$wgScriptPath = "/mediawiki";

## The protocol and server name to use in fully-qualified URLs
$wgServer = "http://localhost";

## The URL path to static resources (images, scripts, etc.)
$wgResourceBasePath = $wgScriptPath;

## The URL paths to the logo.  Make sure you change this from the default,
## or else you'll overwrite your logo when you upgrade!
$wgLogos = [ '1x' => "$wgResourceBasePath/resources/assets/wiki.png" ];

## UPO means: this is also a user preference option

$wgEnableEmail = false;
$wgEnableUserEmail = true; # UPO

$wgEmergencyContact = "apache@🌻.invalid";
$wgPasswordSender = "apache@🌻.invalid";

$wgEnotifUserTalk = false; # UPO
$wgEnotifWatchlist = false; # UPO
$wgEmailAuthentication = true;

## Database settings
$wgDBtype = "sqlite";
$wgDBserver = "";
$wgDBname = "enwikibooks";
$wgDBuser = "";
$wgDBpassword = "";

# SQLite-specific settings
$wgSQLiteDataDir = "/Library/WebServer/data";
$wgObjectCaches[CACHE_DB] = [
    'class' => SqlBagOStuff::class,
    'loggroup' => 'SQLBagOStuff',
    'server' => [
        'type' => 'sqlite',
        'dbname' => 'wikicache',
        'tablePrefix' => '',
        'variables' => [ 'synchronous' => 'NORMAL' ],
        'dbDirectory' => $wgSQLiteDataDir,
        'trxMode' => 'IMMEDIATE',
        'flags' => 0
    ]
];
$wgLocalisationCacheConf['storeServer'] = [
    'type' => 'sqlite',
    'dbname' => "{$wgDBname}_l10n_cache",
    'tablePrefix' => '',
    'variables' => [ 'synchronous' => 'NORMAL' ],
    'dbDirectory' => $wgSQLiteDataDir,
    'trxMode' => 'IMMEDIATE',
    'flags' => 0
];
$wgJobTypeConf['default'] = [
    'class' => 'JobQueueDB',
    'claimTTL' => 3600,
    'server' => [
        'type' => 'sqlite',
        'dbname' => "{$wgDBname}_jobqueue",
        'tablePrefix' => '',
        'variables' => [ 'synchronous' => 'NORMAL' ],
        'dbDirectory' => $wgSQLiteDataDir,
        'trxMode' => 'IMMEDIATE',
        'flags' => 0
    ]
];

# Shared database table
# This has no effect unless $wgSharedDB is also set.
$wgSharedTables[] = "actor";

## Shared memory settings
$wgMainCacheType = CACHE_ACCEL;
$wgMemCachedServers = [];

## To enable image uploads, make sure the 'images' directory
## is writable, then set this to true:
$wgEnableUploads = false;
$wgUseImageMagick = true;
$wgImageMagickConvertCommand = "/usr/local/bin/convert";

# InstantCommons allows wiki to use images from https://commons.wikimedia.org
$wgUseInstantCommons = false;

# Periodically send a pingback to https://www.mediawiki.org/ with basic data
# about this MediaWiki instance. The Wikimedia Foundation shares this data
# with MediaWiki developers to help guide future development efforts.
$wgPingback = false;

## If you use ImageMagick (or any other shell command) on a
## Linux server, this will need to be set to the name of an
## available UTF-8 locale. This should ideally be set to an English
## language locale so that the behaviour of C library functions will
## be consistent with typical installations. Use $wgLanguageCode to
## localise the wiki.
$wgShellLocale = "en_US.UTF-8";

## Set $wgCacheDirectory to a writable directory on the web server
## to make your wiki go slightly faster. The directory should not
## be publicly accessible from the web.
#$wgCacheDirectory = "$IP/cache";

# Site language code, should be one of the list in ./languages/data/Names.php
$wgLanguageCode = "en";

$wgSecretKey = "45fc3d2f7c7896fcf5605ee5572945063d35c36c4ae33972935ed5cd8f311890";

# Changing this will log out all existing sessions.
$wgAuthenticationTokenVersion = "1";

# Site upgrade key. Must be set to a string (default provided) to turn on the
# web installer while LocalSettings.php is in place
$wgUpgradeKey = "a7ee1d991c2b78f0";

## For attaching licensing metadata to pages, and displaying an
## appropriate copyright notice / icon. GNU Free Documentation
## License and Creative Commons licenses are supported so far.
$wgRightsPage = ""; # Set to the title of a wiki page that describes your license/copyright
$wgRightsUrl = "";
$wgRightsText = "";
$wgRightsIcon = "";
$_SERVER['REMOTE_ADDR'] = "127.0.0.1";
    
# Path to the GNU diff3 utility. Used for conflict resolution.
$wgDiff3 = "/usr/bin/diff3";

# The following permissions were set based on your choice in the installer
$wgGroupPermissions['*']['createaccount'] = true;
$wgGroupPermissions['*']['edit'] = true;
$wgGroupPermissions['*']['read'] = true;

## Default skin: you can change the default skin. Use the internal symbolic
## names, ie 'vector', 'monobook':
$wgDefaultSkin = "timeless";

# Enabled skins.
# The following skins were automatically enabled:
wfLoadSkin( 'MonoBook' );
wfLoadSkin( 'Timeless' );
wfLoadSkin( 'Vector' );


# Enabled extensions. Most of the extensions are enabled by adding
# wfLoadExtension( 'ExtensionName' );
# to LocalSettings.php. Check specific extension documentation for more details.
# The following extensions were automatically enabled:
wfLoadExtension( 'CategoryTree' );
wfLoadExtension( 'Cite' );
wfLoadExtension( 'Gadgets' );
wfLoadExtension( 'ImageMap' );
wfLoadExtension( 'InputBox' );
wfLoadExtension( 'LocalisationUpdate' );
wfLoadExtension( 'MultimediaViewer' );
wfLoadExtension( 'OATHAuth' );
wfLoadExtension( 'PageImages' );
wfLoadExtension( 'ParserFunctions' );
wfLoadExtension( 'PdfHandler' );
wfLoadExtension( 'Poem' );
wfLoadExtension( 'Scribunto' );
wfLoadExtension( 'SecureLinkFixer' );
wfLoadExtension( 'SyntaxHighlight_GeSHi' );
wfLoadExtension( 'TemplateData' );
wfLoadExtension( 'TextExtracts' );


# End of automatically generated settings.
# Add more configuration options below.

wfLoadExtension( 'TemplateStyles' );
$wgShowExceptionDetails = true;
wfLoadExtension( 'Math' );
$wgMathValidModes[] = 'mathml';
$wgDefaultUserOptions['math'] = 'mathml';
$wgMathMathMLUrl = 'http://localhost:10042/';

restbase config.yaml (after I followed instructions from https://www.mediawiki.org/wiki/Extension:Math/Restbase and changed some things back to how it is in the example yaml when restbase throws a bunch of errors, possibly due to incompatibility between different versions.)

# RESTBase config for small wiki installs
#
# - sqlite backend
# - parsoid at http://localhost:8142
# - wiki at http://localhost/w/api.php
#
# Quick setup:
# - npm install
#   If you see errors about sqlite, you might have to `apt-get install
#   libsqlite3-dev`.
# - cp config.example.yaml config.yaml
# - double-check and possibly modify lines marked with XXX, then start restbase with
#
#   node server
#
# - If all went well, http://localhost:7231/localhost/v1/page/html/Main_Page
# should show your wiki's [[Main Page]].

services:
  - name: restbase
    module: hyperswitch
    conf:
      port: 7231
      salt: secret
      default_page_size: 125
      user_agent: RESTBase
      ui_name: RESTBase
      ui_url: https://www.mediawiki.org/wiki/RESTBase
      ui_title: RESTBase docs
      spec:
        x-request-filters:
          - path: lib/security_response_header_filter.js
          - path: lib/normalize_headers_filter.js
        x-sub-request-filters:
          - type: default
            name: http
            options:
              allow:
                - pattern: http://localhost/mediawiki/api.php
                  forward_headers: true
                - pattern: http://localhost:8142
                  forward_headers: true
                - pattern: /^https?:\/\//
        paths:
          /{domain:localhost}/{api:v1}:
            x-modules:
              - spec:
                  info:
                    version: 1.0.0
                    title: Wikimedia REST API
                    description: Welcome to your RESTBase API.
                  x-host-basePath: /api/rest_v1
                  x-route-filters:
                    - path: ./lib/normalize_title_filter.js
                      options:
                        redirect_cache_control: '{{options.purged_cache_control}}'
                  paths:
                    /media:
                      x-modules:
                        - path: v1/mathoid.yaml
                          options: '{{options.mathoid}}'
          /{domain:localhost}/{api:sys}:
            x-modules:
              - path: projects/proxy.yaml
                options:
                  backend_host_template: '{{"/{domain}/sys/legacy"}}'
              - spec:
                  paths:
                    /mathoid:
                      x-modules:
                        - path: sys/mathoid.js
                          options: '{{options.mathoid}}'
                    /table:
                      x-modules:
                        - path: sys/table.js
                          options:
                            conf:
                              backend: sqlite
                              dbname: db.sqlite3
                              pool_idle_timeout: 20000
                              retry_delay: 250
                              retry_limit: 10
                              show_sql: false
                              storage_groups:
                                - name: local
                                  domains: /./
                    /legacy/key_value:
                      x-modules:
                        - path: sys/key_value.js
                    /legacy/page_revisions:
                      x-modules:
                        - path: sys/page_revisions.js
                    /post_data:
                      x-modules:
                        - path: sys/post_data.js
                    /action:
                      x-modules:
                        - path: sys/action.js
                          options:
                            apiUriTemplate: http://localhost/mediawiki/api.php
                            baseUriTemplate: http://localhost/mediawiki/api.php/rest_v1
                    /page_save:
                      x-modules:
                        - path: sys/page_save.js
                    /parsoid:
                      x-modules:
                        - path: sys/parsoid.js
                          options:
                            parsoidHost: http://localhost/mediawiki/rest.php
                            response_cache_control: 's-maxage=0, max-age=86400'

# Finally, a standard service-runner config.
info:
  name: restbase

logging:
  name: restbase
  level: info

Restbase config where http://localhost:7231/localhost/v1/ loads successfully:

# RESTBase config for small wiki installs
#
# - sqlite backend
# - parsoid at http://localhost:8142
# - wiki at http://localhost/w/api.php
#
# Quick setup:
# - npm install
#   If you see errors about sqlite, you might have to `apt-get install
#   libsqlite3-dev`.
# - cp config.example.yaml config.yaml
# - double-check and possibly modify lines marked with XXX, then start restbase with
#
#   node server
#
# - If all went well, http://localhost:7231/localhost/v1/page/html/Main_Page
# should show your wiki's [[Main Page]].

services:
  - name: restbase
    module: hyperswitch
    conf:
      port: 7231
      salt: secret
      default_page_size: 125
      user_agent: RESTBase
      ui_name: RESTBase
      ui_url: https://www.mediawiki.org/wiki/RESTBase
      ui_title: RESTBase docs
      spec:
        x-request-filters:
          - path: lib/security_response_header_filter.js
          - path: lib/normalize_headers_filter.js
        x-sub-request-filters:
          - type: default
            name: http
            options:
              allow:
                - pattern: http://localhost/w/api.php
                  forward_headers: true
                - pattern: http://localhost:8142
                  forward_headers: true
                - pattern: /^https?:\/\//
        paths:
          /{domain:localhost}/{api:v1}:
            x-modules:
              - spec:
                  info:
                    version: 1.0.0
                    title: Wikimedia REST API
                    description: Welcome to your RESTBase API.
                  x-route-filters:
                    - path: ./lib/normalize_title_filter.js
                      options:
                        redirect_cache_control: 's-maxage=0, max-age=86400'
                  paths:
                    /page:
                      x-modules:
                        - path: v1/content.yaml
                          options:
                            response_cache_control: 's-maxage=0, max-age=86400'
                        - path: v1/common_schemas.yaml # Doesn't really matter where to mount it.
                    /transform:
                      x-modules:
                        - path: v1/transform.yaml

          /{domain:localhost}/{api:sys}:
            x-modules:
              - path: projects/proxy.yaml
                options:
                  backend_host_template: '{{"/{domain}/sys/legacy"}}'
              - spec:
                  paths:
                    /table:
                      x-modules:
                        - path: sys/table.js
                          options:
                            conf:
                              backend: sqlite
                              dbname: db.sqlite3
                              pool_idle_timeout: 20000
                              retry_delay: 250
                              retry_limit: 10
                              show_sql: false
                              storage_groups:
                                - name: local
                                  domains: /./
                    /legacy/key_value:
                      x-modules:
                        - path: sys/key_value.js
                    /legacy/page_revisions:
                      x-modules:
                        - path: sys/page_revisions.js
                    /post_data:
                      x-modules:
                        - path: sys/post_data.js
                    /action:
                      x-modules:
                        - path: sys/action.js
                          options:
                            apiUriTemplate: http://localhost/mediawiki/api.php
                            baseUriTemplate: "{{'http://{domain}/api/rest_v1'}}"
                    /page_save:
                      x-modules:
                        - path: sys/page_save.js
                    /parsoid:
                      x-modules:
                        - path: sys/parsoid.js
                          options:
                            parsoidHost: http://localhost/mediawiki/rest.php
                            response_cache_control: 's-maxage=0, max-age=86400'

# Finally, a standard service-runner config.
info:
  name: restbase

logging:
  name: restbase
  level: info

methoid config.yaml (also set up according to https://www.mediawiki.org/wiki/Extension:Math/Restbase; passed the curl -d 'q=E=mc^2' localhost:10042 test and the test in the wiki):

# Number of worker processes to spawn.
# Set to 0 to run everything in a single process without clustering.
# Use 'ncpu' to run as many workers as there are CPU units
num_workers: ncpu

# Log error messages and gracefully restart a worker if v8 reports that it
# uses more heap (note: not RSS) than this many mb.
worker_heap_limit_mb: 500

# Logger info
logging:
  level: warn

# Statsd metrics reporter
metrics:
  type: log
#host: localhost
#port: 8125

services:
  - name: mathoid
    # a relative path or the name of an npm package, if different from name
    module: ./app.js
    # optionally, a version constraint of the npm package
    # version: ^0.4.0
    # per-service config
    conf:
      port: 10042
      # interface: localhost # uncomment to only listen on localhost
      # more per-service config settings
      svg: true
      img: true
      texvcinfo: true
      speech: true
      speech_on: false
      speech_config:
        semantics: true
        domain: 'mathspeak'
        style: 'default'
        semantic: false
        minSTree: false
        speakText: true
        enrich: false
      mj_config:
          extensions: "TeX/color,TeX/mediawiki-texvc,TeX/mhchem"
          MathJax:
            menuSettings:
              semantics: true
            SVG:
              font: "TeX"
            TeX:
              noErrors:
                disabled: true
      no_check: true
      dpi: 180
      svgo: false
      # the location of the spec, defaults to spec.yaml if not specified
      # spec: ./spec.template.yaml
      # allow cross-domain requests to the API (default '*')
      cors: '*'
      # to disable use:
      # cors: false
      # to restrict to a particular domain, use:
      # cors: restricted.domain.org
      # content for the CSP headers
      # csp: false  # uncomment this line to disable sending them
      # URL of the outbound proxy to use (complete with protocol)
      # proxy: http://my.proxy.org:8080
      # the list of domains for which not to use the proxy defined above
      # no_proxy_list:
      #   - domain1.com
      #   - domain2.org
      # the list of incoming request headers that can be logged; if left empty,
      # the following headers are allowed: cache-control, content-length,
      # content-type, if-match, user-agent, x-request-id
      # log_header_whitelist:
      #   - cache-control
      #   - content-length
      #   - content-type
      #   - if-match
      #   - user-agent
      #   - x-request-id
      # the user agent to use when issuing requests
      # user_agent: service-template-node
      # the template used for contacting the MW API
      mwapi_req:
        method: post
        uri: http://localhost/w/api.php
        headers:
          host: '{{request.params.domain}}'
          user-agent: '{{user-agent}}'
        body: '{{ default(request.query, {}) }}'
      # the template used for contacting RESTBase
      restbase_req:
        method: '{{request.method}}'
        uri: http://localhost/{{domain}}/v1/{+path}
        query: '{{ default(request.query, {}) }}'
        headers: '{{request.headers}}'
        body: '{{request.body}}'

You must log in to answer this question.

Browse other questions tagged .