Wysiwyg Plugin

Translator framework for Wysiwyg editors

Support for the integration of WYSIWYG (What-You-See-Is-What-You-Get) editors. On its own, the only thing this plugin gives you is a stand-alone HTML to TWiki translator script. For WYSIWYG editing in TWiki, you will also need to install a specific editor package such as TWiki:Plugins.KupuEditorContrib or TWiki:Plugins.WikiwygContrib.

This plugin provides a generic framework that supports editing of TWiki topics using any browser-based HTML editor. It works by transforming TML (TWiki Meta Language) into HTML for the editor, and then transforming HTML back into TML on save.

Features

• Supports the input of malformed HTML
• Full round-trip (TML -> XHTML -> TWiki syntax)
• Framework is editor-agnostic

Details

What's in the package

The package includes the following pieces:

• TML (TWiki syntax) to HTML translator
• HTML to TML translator (with stand-alone script)
• Generic TWiki plugin for automating the translation during editing

How it works

The plugin works by translating the topic text into HTML when someone edits a topic. The HTML is then fed to the WYSIWYG editor. On save, the edited HTML is run through the reverse translation before saving to the topic. TWiki syntax is used in preference to HTML in the stored topic wherever possible, though HTML may be used if the translator can't find a suitable TML equivalent..

The default rendering that TWiki uses to generate HTML for display in browsers is 'lossy' - information in the TWiki syntax is lost in the HTML output, and a round-trip (recovering the original TWiki syntax from the HTML) is impossible. To solve this problem the plugin instead uses its own translation of TWiki syntax to pure XHTML. The generated XHTML is annotated with CSS classes that support the accurate recovery of the original TWiki syntax.

Before you ask the obvious question, yes, the translator could be used to replace the TWiki rendering pipeline for generating HTML pages. In fact, the translator is taken almost directly from the implementation of the rendering pipeline for the TWiki-4 release

Translation of the HTML back to TWiki syntax uses the CPAN:HTML::Parser. This parser is used in preference to a more modern XML parser, because the HTML may not generate fully compliant XHTML. A strict parser would risk losing content. CPAN:HTML::Parser is better at handling malformed HTML.

There is also the advantage that the translator can be used to import HTML from other sources - for example, existing web pages. Due to the simple nature of TWiki syntax and the potential complexity of web pages, this translation is often lossy - i.e there will be HTML features that can be entered by editors that will be lost in this translation step. This is especially noticeable with HTML tables.

Using the translators from Perl scripts

Both translators can be used directly from Perl scripts, for example to build your own stand-alone translators.

A stand-alone convertor script for HTML to TWiki is included in the installation. It can be found in the top-level tools directory and is called html2tml.pl.

Integrating a Wysiwyg Editor

The plugin can be used to generate HTML for an editor in two ways; first, by generating the HTML for the content-to-be-edited directly in the edit template, and second, through a URL that can be used to fetch the content-to-be-edited from the server.

Getting content in the edit template

This is the scenario used by the standard TWiki text editor, except that the text is pre-converted to HTML before inclusion in the template.

The flow of control is as follows:

1. User hits "edit".
2. The beforeEditHandler filters the edit, blocking any attempt to edit restricted content
3. The edit template containing the JS editor is instantiated. The following variables are available for expansion in the template:
   • %WYSIWYG_TEXT% expands to the HTML of the content-to-be-edited. This is suitable for use in a textarea.
   • %JAVASCRIPT_TEXT% expands to the HTML of the content-to-be-edited in a javascript constant.

WYSIWYGPLUGIN_WYSIWYGSKIN must be set for this to work.

Fetching content from a URL

In this scenario, the edit template is generated without the content-to-be-edited. The content is retrieved from the server using a URL e.g. from an IFRAME or using a XmlHttpRequest.

The flow of control is as follows:

1. User hits "edit".
2. If the current skin = WYWIWYGPLUGIN_WYWIWYGSKIN, the beforeEditHandler filters the edit, blocking any attempt to edit restricted content.
3. The edit template containing the JS editor is instantiated.
4. JS editor invokes content URL to obtain the HTML document to be edited
   • The content URL is just a TWiki view URL with the wysiwyg_edit parameter set.
   • The plugin recognises the wysiwyg_edit parameter and uses the TML2HTML? translator to prepare the text, which is then returned as text/plain to the browser.
   • Two TWiki variables, %OWEB% and %OTOPIC%=, should be used in the content URL to refer to the source topic for the content.

Handling Saves

Saves are invoked by the editor POSTing to the TWiki save script with the wysiwyg_edit parameter set to 1. This parameter tells the beforeSaveHandler in the plugin to convert the HTML back to TML. See TWikiScripts for details of the other parameters to the save script.

Once the save script has completed it responds with a redirect, either to an Oops page if the save failed, or to the appropriate post-save URL (usually a view). The editor must be ready to handle this redirect.

Handling Attachments

Attachment uploads can be handled by URL requests from the editor to the TWiki upload script. The upload script normally redirects to the containing topic; a behaviour that you usually don't want in an editor! There are two ways to handle this:

• If the uploads are done in an IFRAME or via XmlHttpRequest, then the 302 redirect at the end of the upload can simply be ignored.
• You can pass noredirect to the upload script to suppress the redirect. In this case you will get a text/plain response of OK followed by a message if everything went well, or an error message if it did not.

Plugin Installation Instructions

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server where TWiki is running.

Like many other TWiki extensions, this module is shipped with a fully automatic installer script written using the BuildContrib.

• If you have TWiki 4.1 or later, and Perl 5.8, you can install from the configure interface (Go to Plugins->Find More Extensions)
  • The webserver user has to have permission to write to all areas of your installation for this to work.
• If you have a permanent connection to the internet (and Perl 5.8), you are recommended to use the automatic installer script
  • Just download the WysiwygPlugin_installer perl script and run it.
• Notes:
  • The installer script will:
    • Automatically resolve dependencies,
    • Copy files into the right places in your local install (even if you have renamed data directories),
    • check in new versions of any installed files that have existing RCS histories files in your existing install (such as topics).
    • If the $TWIKI_PACKAGES environment variable is set to point to a directory, the installer will try to get archives from there. Otherwise it will try to download from twiki.org or cpan.org, as appropriate.
    • (Developers only: the script will look for twikiplugins/WysiwygPlugin/WysiwygPlugin.tgz before downloading from TWiki.org)
  • If you don't have a permanent connection, you can still use the automatic installer, by downloading all required TWiki archives to a local directory.
    • Point the environment variable $TWIKI_PACKAGES to this directory, and the installer script will look there first for required TWiki packages.
      • $TWIKI_PACKAGES is actually a path; you can list several directories separated by :
    • If you are behind a firewall that blocks access to CPAN, you can build a local CPAN mini-mirror, as described at http://twiki.org/cgi-bin/view/Codev/BuildingDakar#CPAN_local_minimirror
• If you don't want to use the installer script, or have problems on your platform (e.g. you don't have Perl 5.8), then you can still install manually:
  1. Download and unpack one of the .zip or .tgz archives to a temporary directory.
  2. Manually copy the contents across to the relevant places in your TWiki installation.
  3. Check in any installed files that have existing ,v files in your existing install (take care not to lock the files when you check in)
  4. Manually edit LocalSite.cfg to set any configuration variables.
  5. Run configure and enable the module, if it is a plugin.
  6. Repeat from step 1 for any missing dependencies.

Plugin Configuration Settings

Translator control

For any of the following controls to work, you must tell WysiwygPlugin the name of the skin being used to invoke the Wysiwyg editor, for example kupu or wikiwyg.

• Set WYSIWYGSKIN =

Note that is can be set differently in different areas by defining WYSWIYGPLUGIN_WYSIWYGSKIN locally (e.g. in WebPreferences).

The global TWiki Variable WYSIWYG_EXCLUDE can be set to make the plugin sensitive to what is in a topic before allowing it to be edited. You can set it up to refuse to edit if

• some or all of HTML tags (e.g. <br /> or <div>), or
• simple variables (e.g. %VAR%) or
• calls (e.g. %VARIABLE{...}%)
• PRE blocks (<pre>)
• HTML comments (<!-- ... -->)

are used in the topic. If the plugin detects an excluded construct in the topic, it will redirect to the default editor. Comma-separated list of one or more of html, variables, calls, pre or comments e.g.

• Set WYSIWYG_EXCLUDE = calls,html

Set WYSIWYG_EXCLUDE in TWikiPreferences, or in WebPreferences for each web.

If you excluded calls in WYSIWYG_EXCLUDE, you can still define a subset of TWiki variables that do not block edits. this is done in the global preference variable WYSIWYG_EDITABLE_CALLS, which should be a list of TWiki variable names separated by vertical bars, with no spaces, e.g:

• Set WYSIWYG_EDITABLE_CALLS = COMMENT|CALENDAR|INCLUDE

Known issues

Incompatible with "non-standard" syntax

WysiwygPlugin is incompatible with plugins that expand non-standard syntax e.g. TWiki:Plugins.MathModePlugin (WysiwygPlugin)

Plugins that extend the syntax using TWiki variables, such as %MYVARIABLE%, should work fine.

Overlapping styles

Because TWiki uses a "best guess" approach to some formatting, it allows overlapping of tags in a way forbidden by HTML, and it is impossible to guarantee 100% that formating in the original TWiki document will still be there when the same document is loaded and then saved through the WysiwygPlugin. The most obvious case of this is to do with styles. For example, the sentence

*bold _bold-italic* italic_

is legal in TML, but in HTML is represented by

<strong>bold <em>bold-italic</em></strong> <em>italic</em>

which gets translated back to TML as

*bold _bold-italic_* _italic_

which is correct by construction, but does not render correctly in TWiki. This problem is unfortunately unavoidable due to the way TWiki syntax works.

Plugin Info

Plugin Authors:	TWiki:Main.CrawfordCurrie http://www.c-dot.co.uk
Copyright	© ILOG 2005 http://www.ilog.fr
License	GPL (Gnu General Public License)
Plugin Version:	12422
Change History:
12422	Added JAVASCRIPT_TEXT to support editors that require topic text in a JS var
12161	Added support for embedded editable HTML in the edit template
12119	Split into WysiwygPlugin and KupuContrib
11538	Minor doc updates, minor fixes to spacing in lists, integrated Koen Marten's template topic patch
9671	Item2025 corrected handling of SPAN and FONT tags used for colour changes
9566	Item1890 doc update
9565	Item1890 Item1041 Item944 Much more aggressive cleanup of HTML pasted in from external sources. Excessively verbose HTML (e.g. from Outlook) was causing apparent infinite looing behaviour.
8867	Item1176 commented out Cairo version of header handler
8780	Item1625 disable expansion of twiki variables in urls where there are other twiki variables that can't be expanded
8779	Item1530 support for templatetopic when editing new topics
8592	Item1532 WysiwygPlugin: Added two more do-not-edit-if-topic-contains parameters, pre+comments
8590	Item1532 WysiwygPlugin: Kenneths suggestion on proper handling of HTML comments (incl. change to kupu)
8572	Item1529 evil, evil. The XMLSerializer in IE isn't happy serializing the DOM. I have no idea why. Kupu manages to get away with this because it passes the DOM through the XML validator, which I had to disable because it strips comments. So, for now, the IE implementation will strip comments - but at least you can save again
8538	Item1501 table handling was a bit spazzy. Several problems fixed.
8535	Item1518 moved icon and string lists into topics, updated screenshot
8531	Item1392 reversed the sense of the navigate-away condition, again
8466	Item1486 added WYSIWYG_EXCLUDE to allow exclusion of 'uneditable' content
8463	Item1486 was stripping comments, wrongly. Had to disable the kupu filters completely, they just do too much damage.
8401	Item1457 corrected problem with bullet list at top of topic
8388	Item1445 fix for a javascript error, introduced by previous fix
8387	Item1445 small usability improvements
8334	Item663 TWiki.org doc merge: Fix incorrect link to kupu website
8327	Item1411 handle case of the result of a TWiki variable being nopped
8312	Item1317 wrong result returned from generation function when expanding HTML embedded in verbatim block
8301	Item1397 removed excess space after sqaub links
8300	Item1231 added %SPAN% to indicate a spanned-over cell in the editor. Improved handling of HTML in verbatim tags by inserting line breaks is the tag type calls for it, before removing the HTML.
8276	Item1215 added WYSIWYG_ICONS and WYSIWYG_TAGS to support user customisation of icon images and twiki variables that can be inserted
8274	Item1314 debugging in case the hang happens again; and made sure to default the editor just in case
8273	Item1315 short forms must be terminated by one of the same characters that terminate wikiwords
8272	Item1391 added special interpretation of IMG tags to expand selected TWiki variables within SRC attributes
8271	Item1340 refined handling of NOP to cover abbrevs
8270	Item1311 removed excess space inserted in headings
8269	Item1339 changed from using arbitrary attribute for notoc to a new CSS class. Arbitrary attributes are stripped by Kupu before save.
8268	Item1344 strip ^Ms inserted by Sarissa during serialisation on IE
8267	Item1394 still can't get text styles to work properly in IE; but I am now firmly of the opinion that the fault lies with the browser, and not with Kupu.
8232	Item1341 added appropriate CSS class
8152	Item1313 added caveat about editing complex HTML and mixed HTML-TML
8151	Item1334 headers not handled properly in Cairo version
8108	Item1318 corrected table/list parser for tables embedded in bulleted lists
8106	Item1310 support for <nop/>
8105	Item1317 support for limited case of nopped variable
8104	Item1320 corrected interpretation of relative URL path in [[]]
8091	Item1259 changed comment handling; rather than trying to create HTML, which gets munged, create an HTML comment. This will only be editable by switching to source view, but hey, it's supposed to be WYSIWYG. Note that this also means that comments in pasted HTML should be retained now
8063	Item1042 spec of SCRIPTURL changed
7904	Item1189 reverting accidental checkin of experimental code
7903	Item1189 filter whitelist is not good enough; need to generate B and I nodes. templates/ pub/TWiki/WysiwygPlugin
7902	Item1189 it took bloody ages to track down, but finally discovered that bold and italic were being filtered out of spans by Kupu 1.3.2.... too smart for it's own good. So added them to the filter whitelist, and it works again.
7873	Item1189 added pre save filter to try and find where the attributes are disappearing to in FF
7872	Item1187 for lack of an s on an RE, the nation was lost (well, the multi-line comment actually). Thanks Kenneth!
7871	Item859 solved issue with non-display of inserted images. Was due to the use of an onSubmit handler to close the dialog, rather than an onLoad handler triggered when the IFRAME that contains the result is loaded.
7869	Item1172 had to rewrite big chunk of the table popup to get it working with 1.3.2
7858	Item1151 rewrote link handlings stuff to leverage browser better
7854	Item1175 escape wikiwords within squabs
7815	Item1158 works for Cairo now as well
7814	Item1158 first implementation of AJAX interface to allow selectoin of topics from other webs
7812	Item1154 removed non-existent scull.gif
7811	Item1155 added extra recursion block, as Item1155 suggests it is needed
7801	Item1042 All sorts of clever tricks to handle expansion/compression of a subset of TWiki variables when they are used in URLs. Not a complete solution, but better than it was.
7799	Item1024 caught out by recursive call to beforeCommonTagsHandler in Cairo (nasty)
7798	Item1042 whoops, broke \t conversion in Cairo
7789	Item1140 testcase for 1140
7788	Item1140 fix rewriting of img src urls (and updated MANIFEST for Kupu1.3.2)
7786	Item1042 extensive improvements to variable and URL recognition and conversion
7766	Item856 added doc on EDIT_SKIN to the plugin
7712	Item1074 upgrade to Kupu 1.3.2 complete (at last)
7710	Item1074 Fixed source edit mode
7709	Item1074 tidied up broken toolbar. There are still known issues
7700	Item1074 first pass at moving to Kupu 1.3.2.
7673	Item1037 insert wikiword only if selection is zero length
7672	Item977 changed to remove dangerous Cairo-based assumption, and use context ids instead
7630	Item1025 added 'escape clause' for old handlers implemented to support old TWiki releases without warnings
7506	Item941 Eliminated the last of the dynamic globals to try and solve saving problem. Can;t test with mod_perl, but is fine with speedycgi AFAICT
7456	Item873 minor issue; replace br with \n in pre
7455	Item873 obvious problem parsing closing pre tag on same line as open tag
7453	Item710 Handling HTML comments
7452	Item876 Item945: Item876: spacing around table cells, correct handling of variables. Had to compromise on handling [[]] but I think it's for the best.
7430	Item871 made sure that brackets are generated for non-wikiwords
7425	Item928 removed special interpretation of mailto links
7424	Item866 extended URL parsing to handle MAINWEB and TWIKIWEB twiki variables, in the same hacky way as the core.
7416	Item870 a couple of corner-cases for correct handling of twiki variables
7401	Item899 changed list generation to use spaces instead of tabs
7265	Item180 removed pointless, outdated dependency check from DateFieldPlugin?
6935	Item622 reverted 3 specs to tabs in Set lines in plugins topics for kompatterbility with Kigh-roe
6905	Item622 tabs -> 3 spacesto avoid confusing the users
6850	Item638 added instruction to run configure to all install docs (I hope)
6827	Item569 added default RELEASE to everything that had a version, and removed a load of dead code that was getting in the way
6758	Item569 computed version numbers for plugins from the repository rev they were built from.
6504	Item436 incremented vernos of all changed plugins
6485	Item429 trying to make access controls clearer
6401	Item340 re-initialisation bug found by ColasNahaboo? when using mod_perl; fixed by correctly re-initialising the parse stack for each run of the convertor
6284	Item340 Release 0.16 of WysiwygPlugin
6279	Item340 bugfixes for release 0.16 of WysiwygPlugin
6261	Item335 Switched PNGs to indexed mode, as transparency doesn't work on IE for RGB images
6238	Item332 Added context identifier to WysiwygPlugin, and a button to the pattern view template. If WysiwygPlugin is enabled, then the button will appear. Neat, huh?
6195	Item196 getting plugin test suites to pass. Doesn't mean the plugins actually work, just that the test suites run (which is a good indicator)
6174	Item168 checkpoint checking for 0.16
6151	Item186 more minor updates
6150	Item168 new icons, and a couple of bugfixes, to WysiwygPlugin
6092	Item196 more plugin and contrib fixes for develop; mainly just moving tests around and making sure they all pass.
6067	Item138 had to change to using beforeCommonTagsHandler and also escape % signs to prevent TWiki from rendering internal tags (as reported by Colas)
5979	Item168 corrected stupid error on IE; added screenshot
5977	Item168 release 0.13
5948	Item168 nearly ready for 0.13
5937	Item168 corrected images, twikified all images
5936	Item168 the import from cvs has screwed images
5934	Item168 twikified icon images, and renamed some images to be more intention-revealing
5739	0.12 beta release
5730	V0.11
5714	Tidied up installer, documentation. Release 0.10
5712	pre-release 0.06
5706	Version 0.05
5705	Checkpoint checking - version 0.03
5702	cvsrmtee old files
5701	Check in for prototype release
5700	Check in for prototype release
5699	Checkpoint
5698	Most of the toolboxes are working again
5693	Initial commit; doesn't do much except run tests
Dependencies:	Name Version Description HTML::Parser >=3.28 Required. Available from CPAN. HTML::Entities >=1.25 Required. Available from CPAN.
Plugin Home:	http://twiki.org/cgi-bin/view/Plugins/WysiwygPlugin
Feedback:	TWiki:Plugins/WysiwygPluginDev
Appraisal:	http://twiki.org/cgi-bin/view/Plugins/WysiwygPluginAppraisal

Related Topics: TWikiPreferences, TWikiPlugins