Add python3 support. Improve webserver logic using bottle

instead of flask and ajax long polling instead of websockets.
Improve debugging. Improve tox build and tests.
Reduce and update dependencies.

.travis.yml

@@ -7,11 +7,11 @@ matrix:
       osx_image: beta-xcode6.3
 
 python:
-  - 2.7
+  - 3.4
 
 install:
-  - sudo bash -c 'python2.7 <(curl https://bootstrap.pypa.io/get-pip.py)'
-  - sudo pip install tox
+  - sudo bash -c 'python3.4 <(curl https://bootstrap.pypa.io/get-pip.py)'
+  - sudo pip3 install tox
 
 script:
   - tox

README.rst

@@ -41,17 +41,29 @@ Design considerations
 sphinxserve was originally conceived as a Python and Linux project that can
 visualize sphinx document modifications in real time while working on them. At
 its core, sphinxserve uses the awesome projects `gevent`_  to provide
-concurrency and event coordination, `flask`_ for web communication, Sphinx
-for reStructucturedText rendering and of course `Python`_. sphinxserve used to
-control browser reloading with xdotool, introducing a complex system tool
-dependency. On release 0.7, sphinxserve decouples from this system dependency
-using instead flask-sockets python package. The tradeoff here was to
-temporarily drop python3 support until the gevent ecosystem officially
-supports python3 which should be soon. Also, the filesystem notification tool
-was upgraded to watchdog, removing another system dependency and making
-the code more generic and cleaner. With these changes, as of release 0.7.4,
-sphinxserve is able to run in other platforms as OSX and Windows for example.
+concurrency and event coordination, `bottle`_ for web communication,
+`watchdog`_ for filesystem events, `Sphinx`_ for reStructucturedText rendering
+and of course `Python`_.
 
+History
+=======
+
+release 0.8: sphinxserve fully supports python3. bottle replaces flask and
+ajax long polling replaces websockets to simplify even more the web server
+logic.
+
+release 0.7.4: sphinxserve is able to run in other platforms as OSX and Windows
+for example.
+
+release 0.7: sphinxserve decoupled from xdotool using flask-sockets python
+package. The tradeoff was to temporarily drop python3 support until the gevent
+ecosystem officially supported python3. Also, the filesystem notification tool
+was upgraded to watchdog, removing another system dependency and making the
+code more generic and cleaner.
+
+release <0.7: sphinxserve used to control browser reloading with xdotool,
+a complex system tool dependency only available on Unix systems and tested
+on Linux.
 
 Installation
 ============
@@ -77,34 +89,35 @@ Ubuntu>=14, Centos>=7 and Arch distros on Linux and in Yosemite on OSX.
 Linux
 -----
 
-System dependencies: glibc linux>=3, python>=2.7,<3 and a web browser
+System dependencies: glibc linux>=3, python>=2.7 and a web browser
 supporting websockets (Firefox, Chrome, etc) on Linux::
 
-    $ wget -O ~/bin/sphinxserve https://github.com/mzdaniel/sphinxserve/releases/download/0.7.4/sphinxserve-linux
+    $ wget -O ~/bin/sphinxserve https://github.com/mzdaniel/sphinxserve/releases/download/0.7.5/sphinxserve-linux
     $ chmod 755 ~/bin/sphinxserve
 
 OSX
 ---
 
 Yosemite already has all needed dependencies::
 
-    $ wget -O ~/bin/sphinxserve https://github.com/mzdaniel/sphinxserve/releases/download/0.7.4/sphinxserve-osx
+    $ wget -O ~/bin/sphinxserve https://github.com/mzdaniel/sphinxserve/releases/download/0.7.5/sphinxserve-osx
     $ chmod 755 ~/bin/sphinxserve
 
 
 Python package
 ~~~~~~~~~~~~~~
 
-Linux system dependencies: glibc linux>=3, python>=2.7,<3, the C toolchain
+Linux system dependencies: glibc linux>=3, python>=2.7, the C toolchain
 (package names dependent on linux distro) to compile gevent and a web browser
-supporting websockets. pip automatically downloads sphinxserve and its python
+supporting javascript. pip automatically downloads sphinxserve and its python
 dependencies, compiles and builds wheel binary packages as needed and finally
 install sphinxserve.
 
-OSX system dependencies: Xcode. Verified to work on Yosemite.
+OSX system dependencies: Verified to work on Yosemite, python >=2.7 and
+a web browser supporting javascript ajax with just pip installing.
 
-Windows system dependencies: Verified to work on Windows 7, python >=2.7,<3 and
-a web browser supporting websockets with just pip installing.
+Windows system dependencies: Verified to work on Windows 7, python >=2.7 and
+a web browser supporting javascript ajax with just pip installing.
 
 In all systems::
 
@@ -157,14 +170,29 @@ your browser to localhost:8888. Any saved changes on rst or txt files will
 trigger docs rebuild.
 
 
+Local test/build
+================
+
+Assumptions for this section: A unix system, python2.7, 3.4 or 3.5, and
+pip >= 8.1. Although git is recommended, it is not required.
+
+We use tox to test sphinxserve in virtualenvs for python2.7, 3.4 and 3.5
+Tox is a generic virtualenv manager and test command line tool. It handles the
+creation of virtualenvs with proper python dependencies for testing, pep8
+checking and building:
+
+    $ git clone https://github.com/mzdaniel/sphinxserve; cd sphinxserve
+    $ pip install tox
+    $ tox
+
+
 Thanks!
 =======
 
 * `Guido van Rossum`_ and `Linus Torvalds`_
 * Georg Brandl & David Goodger for `Sphinx`_ and `reStructuredText`_
 * Denis Bilenko, Armin Rigo & Christian Tismer for `Gevent`_ and `Greenlet`_
-* Armin Ronacher for `Flask`_
-* Jeffrey Gelens & Kenneth Reitz for `gevent websocket`_ and `flask sockets`_
+* Marcel Hellkamp for `bottle`_
 * Yesudeep Mangalapilly for `watchdog`_
 * Holger Krekel for `pytest`_ and `tox`_
 * Eric Holscher for `Read The Docs`_
@@ -177,13 +205,11 @@ Thanks!
 .. _Guido van Rossum: http://en.wikipedia.org/wiki/Guido_van_Rossum
 .. _Linus Torvalds: http://en.wikipedia.org/wiki/Linus_Torvalds
 .. _python: https://www.python.org
-.. _sphinx: http://sphinx-doc.org/tutorial.html
+.. _Sphinx: http://sphinx-doc.org/tutorial.html
 .. _restructuredtext: http://docutils.sourceforge.net/rst.html
 .. _gevent: http://gevent.org
 .. _greenlet: https://github.com/python-greenlet/greenlet
-.. _flask: http://flask.pocoo.org
-.. _gevent websocket:  https://bitbucket.org/Jeffrey/gevent-websocket
-.. _flask sockets: https://github.com/kennethreitz/flask-sockets
+.. _bottle: http://bottlepy.org/docs/dev/index.html
 .. _watchdog: https://github.com/gorakhargosh/watchdog
 .. _pytest: http://pytest.org
 .. _pex: https://github.com/pantsbuild/pex

pex/setup.cfg

@@ -32,4 +32,5 @@ classifier =
 
 [files]
 packages = sphinxserve_pex
-scripts =  scripts/sphinxserve
+data_files =
+    bin = scripts/sphinxserve

requirements.txt

@@ -1,5 +1,5 @@
-flask-sockets>=0.1
-gevent>=1.1b2
-loadconfig>=0.1
+bottle==0.12.9
+gevent>=1.1.1
+loadconfig>=0.1.1
 sphinx>=1.2.3
 watchdog>=0.8.3

scripts/sphinxserve

@@ -0,0 +1,9 @@
+#!/usr/bin/env python
+
+import gevent.monkey
+gevent.monkey.patch_all()
+
+from sphinxserve import main
+import sys
+
+main(sys.argv)

setup.cfg

@@ -20,8 +20,8 @@ classifier =
     Intended Audience :: System Administrators
     License :: OSI Approved :: MIT License
     Programming Language :: Python
-    Programming Language :: Python :: 2
     Programming Language :: Python :: 2.7
+    Programming Language :: Python :: 3
     Topic :: Documentation
     Topic :: Documentation :: Sphinx
     Topic :: Text Processing :: Markup

sphinxserve/__init__.py

@@ -13,12 +13,12 @@
 from gevent.event import Event
 from gevent import spawn, joinall
 from loadconfig import Config
-from loadconfig.lib import capture_stream, write_file
+from loadconfig.lib import write_file
 import logging as log
 import os
 from os.path import exists
 from sphinx import build_main
-from sphinxserve.lib import fs_event_ctx, Webserver
+from sphinxserve.lib import capture_streams, fs_event_ctx, Webserver
 import sys
 from textwrap import dedent
 
@@ -128,26 +128,24 @@ def render(self):
         while True:
             self.watch_ev.wait()  # Wait for docs changes
             self.watch_ev.clear()
-            with capture_stream() as stdout:
+            with capture_streams() as streams:
                 self.build()
-            log.debug(stdout.getvalue())
+            log.debug(streams.getvalue())
             self.render_ev.set()
 
     def build(self):
-        return build_main([
-            'sphinx-build',
-            self.c.sphinx_path,
-            os.path.join(self.c.sphinx_path, self.c.output)
-        ])
+        '''Render reStructuredText files with sphinx'''
+        return build_main(['sphinx-build', self.c.sphinx_path,
+            os.path.join(self.c.sphinx_path, self.c.output)])
 
     def manage(self):
         '''Manage web server, watcher and sphinx docs renderer
         '''
-        with capture_stream() as stdout, capture_stream('stderr') as stderr:
+        with capture_streams() as streams:
             ret = self.build()
         if ret != 0:
-            sys.exit(stderr.getvalue())
-        log.debug(stdout.getvalue())
+            sys.exit(streams.getvalue())
+        log.debug(streams.getvalue())
         workers = [spawn(self.serve), spawn(self.watch), spawn(self.render)]
         joinall(workers)
 

sphinxserve/lib.py

@@ -5,78 +5,69 @@
 import os
 os.read = tp_read
 
+from bottle import get, run, static_file
 from contextlib import contextmanager
-from flask import Flask
-from flask_sockets import Sockets
 import gevent
-from gevent.pywsgi import WSGIServer
 from gevent.queue import Queue
 from gevent import sleep
-from geventwebsocket.handler import WebSocketHandler
 from loadconfig.lib import Ret
+from loadconfig.py6 import cStringIO
 import re
 import socket
-from sys import platform
+import sys
 from textwrap import dedent
 from watchdog.events import PatternMatchingEventHandler
 from watchdog.observers import Observer
 
-if platform.startswith('win') or platform.startswith('darwin'):
+if sys.platform.startswith('win') or sys.platform.startswith('darwin'):
     from watchdog.observers.polling import PollingObserver as Observer  # noqa
 
 
 class Webserver(object):
     '''Serve static content from path, featuring asynchronous reload.
-    reload uses flask-sockets for async gevent communication on the server
-    and websockets on the browser.
+    Page reload is triggered by sphinx rst updates using gevent on the server
+    and executed after ajax long polling on the browser.
     '''
-    def __init__(self, path, host, port, sig_reload):
-        self.path = path
-        self.host = host
-        self.port = port
-        self.signal = sig_reload
+    def __init__(self, path, host, port, reload_ev):
+        self.path, self.host, self.port = path, host, port
+        self.reload_ev = reload_ev
 
     def run(self):
         reload_js = dedent('''\
         <script type="text/javascript">
             $.ajaxSetup({cache: false})  // drop browser cache for refresh
-            var ws = new WebSocket("ws://" + location.host + "/ws")
-            ws.onclose = function() {    // reload server signal
-                window.location.reload(true)}
+            $(document).ready(function() {
+                $.ajax({ type: "GET", async: true, cache: false,
+                    url: location.protocol + "//" + location.host + "/_svwait",
+                    success: function() {window.location.reload(true)} }) })
         </script>''')
-        app = Flask(
-            __name__,
-            static_url_path='',
-            static_folder=self.path
-        )
-        sockets = Sockets(app)
-
-        @app.route('/')
-        def root():
-            return app.send_static_file('index.html')
-
-        @app.after_request
+
         def after_request(response):
             '''Add reload javascript and remove googleapis fonts'''
-            response.direct_passthrough = False
-            if response.content_type.startswith('text/html'):
-                response.data = re.sub('(</head>)', r'{}\1'.format(reload_js),
-                    response.data, flags=re.IGNORECASE)
-            if response.content_type.startswith('text/css'):
-                response.data = re.sub(
-                    '@import url\(.+fonts.googleapis.com.+\);', '',
-                    response.data, flags=re.IGNORECASE)
-            return response
-
-        @sockets.route('/ws')
-        def ws_socket(ws):
-            '''Reload browser'''
-            self.signal.wait()
-            self.signal.clear()
-            ws.close()
-
-        WSGIServer((self.host, int(self.port)), app,
-            handler_class=WebSocketHandler).serve_forever()
+            r = response
+            r.body = r.body.read().decode('utf-8') if getattr(
+                r.body, 'read', False) else r.body
+            if r.content_type.startswith('text/html'):
+                r.body = re.sub('(</head>)', r'{}\1'.format(reload_js),
+                    r.body, flags=re.IGNORECASE)
+            if r.content_type.startswith('text/css'):
+                r.body = re.sub('@import url\(.+fonts.googleapis.com.+\);', '',
+                    r.body, flags=re.IGNORECASE)
+            return r
+
+        @get('<path:path>')
+        def serve_static(path):
+            path = path + '/index.html' if path.endswith('/') else path
+            response = static_file(path, root=self.path)
+            return after_request(response)
+
+        @get('/_svwait')
+        def wait_server_event():
+            '''Block long polling javascript until reload event'''
+            self.reload_ev.wait()
+            self.reload_ev.clear()
+
+        run(host=self.host, port=int(self.port), server='gevent')
 
 
 @contextmanager
@@ -110,6 +101,25 @@ def fs_event(self):
     del evh
 
 
+@contextmanager
+def capture_streams():
+    r'''Capture streams (stdout & stderr) in a string
+    >>> with capture_streams() as streams:
+    ...     print('Hi there')
+    >>> streams.getvalue()
+    'Hi there\n'
+    '''
+    stdout = sys.stdout
+    stderr = sys.stderr
+    data = cStringIO()
+    sys.stdout = data
+    sys.stderr = data
+    yield data
+    sys.stdout = stdout
+    sys.stderr = stderr
+    data.flush()
+
+
 class Timeout(gevent.Timeout):
     '''Add expired attribute to Timeout context manager'''
     def __init__(self, *args, **kwargs):

tests/test_requirements.txt

@@ -1,3 +1,3 @@
-hacking==0.10.1
-pytest==2.7.0
-requests==2.7.0
+flake8==2.5.4
+pytest==2.9.0
+requests==2.9.1