Wikitech-l May 2016

113 participants
123 discussions

ResourceLoader addModuleStyles() issues
by Krinkle 10 May '16

10 May '16

TL;DR: The current addModuleStyles() method no longer meets our requirements. This mismatch causes bugs (e.g. user styles load twice). The current logic also doesn't support dynamic modules depending on style modules. I'm looking for feedback on how to best address these issues. ResourceLoader is designed with two module types: Page style modules and dynamic modules. Page style modules are part of the critical rendering path and should work without JavaScript being enabled or supported. Their URL must be referenced directly in the HTML to allow browsers to discover them statically for optimal performance. As such, this URL can't use dynamic information from the startup module (no dependencies[1], no version hashes). Dynamic modules have their names listed in the page HTML. Their dependencies and version hashes are resolved at run-time by JavaScript via the startup manifest. We then generate the urls and request them from the server. There is also a layer of object caching in-between (localStorage) which often optimises the module request to not involve an HTTP request at all. Below I explain the two issues, followed by a proposed solution. ## Dependency Historically there was no overlap between these two kinds of modules. Page style modules were mostly dominated by the skin and apply to skin-specific server-generated html. Dynamic modules (skin agnostic) would make their own DOM and apply their own styles. Now that we're reusing styles more, we're starting seeing to see overlap. In the past we used jQuery UI - its styles never applied to PHP-generated output. Now, with OOUI, its styles do apply to both server-side and client-side generated elements. Its styles are preloaded as a page style module on pages that contain OOUI widgets. The OOjs UI bundle (and its additional styles) shouldn't need to know whether the current page already got some of the relevant styles. This is what dependencies are for. For OOUI specifically, we currently have a hardcoded work around that adds a hidden marker to pages where OOUI is preloaded. The OOUI style module has a skipFunction that forgoes loading if the marker is present. ## Implicit type Aside from the need for dependencies between dynamic and page style modules. There is another bug related to this. We don't currently require modules to say whether they are a dynamic module or a page style module. In most cases this doesn't matter since a developer would only load it one way and the module type is implied. For example, if you only ever pass a module to addModules() or mw.loader() it is effectively a dynamic module. If you only ever pass a module to addModuleStyles() loaded via a <link rel=stylesheet> then it is a page style module. A problem happens if one tries to load the same module both ways. This might seem odd (and it is), but happens unintentionally right now with wiki modules (specifically, user/site modules and gadgets). For user/site modules, we don't know whether common.css relates to the page content, or whether it relates to dynamic content produced by common.js. The same for gadgets. A gadget may produce an AJAX interface and register styles for it, or the gadget may be styles-only and intended as a skin customisation. Right now the Gadgets extension works around this problem with a compromise: Load it both ways. First it loads all gadgets as page style modules (ignoring the script portion), and then it loads the same modules again as dynamic modules. Thus loading the styles twice. ## Proposed solution In order to allow dependency relationships between a dynamic module and a page style module, we need to inform mw.loader in JavaScript about which modules have already been loaded by different means. We do this already with the user modules (by setting mw.loader.state directly). This would work but means that if you then load the same module again as dynamic module, it will assume the module is already loaded and thus never deliver the script portion (for the case where the gadget wasn't meant to be a page style module). Similarly, it would mean that common.js wouldn't get delivered if common.css exists. For user/site modules I propose to solve this by splitting them up into 'user', 'user.styles', 'site' and 'site.styles'. Existing dependency relationships between other modules and 'user' or 'site' will continue to work. It'd mostly be an internal detail. This allows us to load one as a page style module and the other dynamically. For gadgets we can try to infer the intent (styles-only = page style module, both = dynamic module), with perhaps a way to declare the desired load method explicitly in Gadgets-definition if the default is wrong. With that resolved, we can export mw.loader.state information for page style modules, which then allows dynamic modules to depend on them. Thoughts? https://phabricator.wikimedia.org/T87871 https://phabricator.wikimedia.org/T92459 -- Krinkle [1] If one would allow page style modules to have dependencies and resolve them server-side in the HTML output, this would cause corruption when the relationship between two modules changes as existing pages would have the old relationship cached but do get the latest content from the server. Adding versions wouldn't help since the server can't feasibly have access to previous versions (too many page/skin/language combinations).

5 13

historical trivia: who first picked UTC as Wikimedia time, when and why?
by David Gerard 10 May '16

10 May '16

Question about obscure historical detail: Who picked UTC as Wikimedia time? When was this, and what was the thought process? (the answer is almost certainly "Brion or Jimbo, early 2001, it's the obvious choice", but I'm just curious as to details.) - d.

6 5

MediaWiki codesniffer updates on extensions
by Niklas Laxström 10 May '16

10 May '16

I noticed that lots of extensions were recently updated to use mediawiki-codesniffer 0.7.1 [1]. This has some implications I think are worth being aware of: 1) Many open patches will need manual rebasing and conflict resolution. 2) Those extensions will now depend on at least PHP 5.4 due to short array syntax [2] instead of PHP 5.3. Both can cause inconveniences, (1) mainly for developers and (2) mostly for users. Some patches doing this update in a more thorough and tested way [3] existed but were ignored. It would be nice to avoid that in the future. Please also watch out for code that might have got removed unexpectedly due to a bug [4]. On quick inspection I did not see this happening in those patches. [1] https://gerrit.wikimedia.org/r/#/q/status:merged+topic:bump-dev-deps,n,z [2] https://secure.php.net/manual/en/migration54.new-features.php [3] For example https://gerrit.wikimedia.org/r/#/c/287315/ [4] https://phabricator.wikimedia.org/T134857 -Niklas

1 0

JetBrains licenses for Volunteers and Staff
by Yuri Astrakhan 10 May '16

10 May '16

PhpStorm, InteliJ IDEA, Resharper and other JetBrain users, we just received free upgraded licenses for all of their products. Check your account at https://account.jetbrains.com/licenses and login with that account inside your application to automatically use that license. If you don't see the license, contact me or Sam Reed, and we will add you right away. On IRC: yurik or reedy, or via email. We could also use a few more admins for these licenses. You will no longer need to copy/paste any licenses manually. IDEA: use for PHP, JavaScript, Puppets, Ruby, and other web-related technologies https://www.jetbrains.com/idea/ Resharper: for C# and Microsoft Visual Studio C++ https://www.jetbrains.com/resharper/ CLion: for C++ https://www.jetbrains.com/clion/

4 6

Help improve search relevance with Discernatron 🤖
by Chris Koerner 09 May '16

09 May '16

The Discovery department's [1] work to improve search continues with a new tool! We are asking volunteers to help choose, or discern, which search results are the most relevant. One way of improving search results relevance is to provide search results from multiple search engines side-by-side for comparison. Participants pick the best, most relevant results, which are then used to tune our own search results. It's one way to help improve search with human assistance, by showing articles that are most relevant to search queries. This system is used by many R&D departments and gives great results. Discernatron is a tool developed by the Discovery department for just this sort of work. Visitors are asked to pick the most relevant results across four different search result sets. The data is then used to help improve our relevancy model for search. Screenshot at [2] If you are interested in helping, you can access Discernatron at https://discernatron.wmflabs.org/ and authenticate with your unified user account. To learn more about the tool visit Mediawiki.org. [3] [1] https://www.mediawiki.org/wiki/Discovery [2] https://meta.wikimedia.org/wiki/File:Discernatron_screenshot.png [3] https://www.mediawiki.org/wiki/Discernatron -- Yours, Chris Koerner Community Liaison - Discovery Wikimedia Foundation

1 0

Thursday: Get your patch reviewed during "Code Review office hours"
by Mukunda Modell 09 May '16

09 May '16

Starting Thursday May 12th, 13:00 PDT ( 20:00 GMT ) we will be having the first weekly Code Review office hours on freenode IRC in the #wikimedia-codereview channel. Event details: https://phabricator.wikimedia.org/E179 Background: https://phabricator.wikimedia.org/T128371 Thanks to everyone who's been helping to organize this. We would welcome people to submit your patches for review as well as reviewers who can spare a few minutes to provide feedback and hopefully merge some patches! If you can't make it during the scheduled time period then please feel free to suggest other times that would be better for you. I intend to set up one or two other weekly time slots, at least one of which should be at a time that's more convenient for people in Europe and Asia. Looking forward to seeing you in #wikimedia-codereview ______________________ Mukunda Modell, Release Engineer Wikimedia Foundation

2 1

Re: [Wikitech-l] garbage characters show up when fetching wikimedia api
by MZMcBride 07 May '16

07 May '16

Trung Dinh wrote: >Hi all, >I have an issue why trying to parse data fetched from wikipedia api. >This is the piece of code that I am using: >api_url = 'http://en.wikipedia.org/w/api.php' >api_params = >'action=query&list=recentchanges&rclimit=5000&rctype=edit&rcnamespace=0&rc >dir=newer&format=json&rcstart=20160504022715' > >f = urllib2.Request(api_url, api_params) >print ('requesting ' + api_url + '?' + api_params) >source = urllib2.urlopen(f, None, 300).read() >source = json.loads(source) > >json.loads(source) raised the following exception " Expecting , >delimiter: line 1 column 817105 (char 817104" > >I tried to use source.encode('utf-8') and some other encodings but they >all didn't help. >Do we have any workaround for that issue ? Thanks :) Hi. Weird, I can't reproduce this error. I had to import the "json" and "urllib2" modules, but after doing so, executing the code you provided here worked fine for me: <https://phabricator.wikimedia.org/P3009>. You probably want to use 'https://en.wikipedia.org/w/api.php' as your end-point (HTTPS, not HTTP). As far as I know, JSON is always encoded as UTF-8, so you shouldn't need to encode or decode the data explicitly. The error you're getting generally means that the JSON was malformed for some reason. It seems unlikely that MediaWiki's api.php is outputting invalid JSON, but I suppose it's possible. Since you're coding in Python, you may be interested in a framework such as <https://github.com/alexz-enwp/wikitools>. MZMcBride

3 3

Ajaxify page requests extension
by Cyken Zeraux 07 May '16

07 May '16

I've been working on a little project that intercepts normal wiki links from reloading the entire page and instead requests that page with ajax, seperating out the content and replacing it on the page. So far, I've gotten this to work with only the main content, however I need to also be able to safely grab the 'mw-navigation' and 'footer' which seems to also be standardized in most skins. The only problem with that is client side JS cannot safely parse returned pages for all three elements because it is possible for users to specify replica elements with the same ID in-page, and DomDocument is not optimized or well supported. I would like to be able to grab all of the page content before its echoed out, DomDocument in PHP, grab my elements, and echo out JSON of them. The client side js plugin would then refresh those elements. I've done a good bit of trial and error and research in the Mediawiki docs for this, and it seems its not currently supported because the active skin is the one that echoes out the entire page, not Mediawiki itself. Am I wrong in my findings, and are there any plans to make Mediawiki handle pages and echo them out? It would break standard currently, but I feel that having skins build up an output string and passing it to Mediawiki rather than delivering the content itself is a better approach. Thank you for your time.

2 1

Diff algorithms: the shootout
by Max Semenik 07 May '16

07 May '16

Right now, MediaWiki has 2 pure-PHP engines to produce diffs (there's also a native PHP extension wikidiff2, but we're not discussing it right now): * DairikiDiff is what everybody uses, and * Wikidiff3, and alternative implementation by Guy Van den Broeck that was around for 8 years but required a configuration change While less battle-tested, Wikidiff3 offers vastly improved performance on heavy diffs compared to DairikiDiff. The price, however, is that it makes certain shortcuts if the diff is too complex. I ran through 100K diffs from English Wikipedia, and 6% of diffs were different. Lots of changes were seemingly insignificant but I need your help with determining if it's really so. I've built this tool <https://diff-forge.wmflabs.org/wiki/Special:DiffCompare>[1] to facilitate the comparison. It displays two diffs from different algorithms side by side (yeah, it can get too wide, I know:P). Which of them is which is random. Parts with differences between the implementations are highlighted in yellow. Below is the diff of differences for the reference. You can vote with buttons above the diffs, no registration is required. If you see a catastrophically bad diff please send me the link. Unless the results are significantly worse, I'd like to go ahead and make wikidiff3 the only implementation. [1] https://diff-forge.wmflabs.org/wiki/Special:DiffCompare -- Best regards, Max Semenik ([[User:MaxSem]])

6 6

Discovery Weekly Update for the week starting 2016-05-02
by Chris Koerner 06 May '16

06 May '16

Greetings, I hope you had a good week. Here are the updates from the Discovery department for the week. * Wikipedia.org portal A/B test for adding in descriptive text to the sister wiki links <https://phabricator.wikimedia.org/T131238> went live on May 3, 2016. * Updated statistics for wiktionary.org <https://phabricator.wikimedia.org/T128546> on May 3, 2016 * Released a few minor fixes to the Portal on May 3, 2016 * New version of Blazegraph and WDQS deployed, including geospatial search <https://www.mediawiki.org/wiki/Wikidata_query_service/User_Manual#Geospatia…> . * New completion suggester code was reverted from planned ElasticSearch 2.x release <https://lists.wikimedia.org/pipermail/discovery/2016-May/001085.html>, affecting our plans for runtime updates to suggester. * It is now possible to use SPARQL queries <https://phabricator.wikimedia.org/T126741> from interactive graphs <https://www.mediawiki.org/wiki/Extension:Graph/Demo>. * Deb attended the Reading department offsite this week. Initial response was that it was a great opportunity to learn and discovery new ways we can collaborate in the future. ---- Feedback and suggestions on this weekly update are welcome. The full update, and archive of past updates, can be found on Mediawiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates -- Yours, Chris Koerner Community Liaison - Discovery Wikimedia Foundation

1 0

← Newer
1
...
7
8
9
10
11
12
13
Older →

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l May 2016