The Httpd Module and Toadhttpd

Sean Woods

Test & Evaluation Solutions, LLC

(Press the right arrow on your keyboard to advance the slides or you can just watch Sean present it himself on YouTube)

• Why we need a reference web server
• The Httpd Module in Tcllib
• The Toadhttpd web server project

Folks at home can tune into:

http://www.etoyoc.com/yoda/papers/tcl2018/Httpd_Slides.html

Why we need a reference web server

Why we need a reference web server

• Tcl needs to explore new ways of generating human machine interfaces.
• Newer platforms are stressing browser based interactions.
• Especially on mobile platforms.

Why we need a REFERENCE web server

The Tcl community has already developed a number of web servers.

The problem is:

• The serious implementations are too massive, complex, and/or needy to be embedded in a desktop application.
• The embeddable implementations are basically toys.
• The implementations which support server side scripting, each go off an implement server side scripting in a completely different way.
• Testing. Testing. Testing.

The Httpd Module

• Started off life as version 4 of Tclhttpd.
• Introduced Coroutines and TclOO to the core of the httpd server
• Added bootstrap and jquery integration
• Added a highly adaptable dispatch engine
• Server was implemented as an OO object
• Could be loaded in a package inside of normal Tcl software

Ended up being nothing like Tclhttpd.

Simple Dustmote Example:

package require httpd

::httpd::server create HTTPD port 8015 doc_root ~/htdocs
HTTPD start

There are three players: The Server, the Coroutine, and the Reply Object

The players

The process starts when a socket event invokes a call to the server's connect method.

The players

The connect method creates a coroutine which takes control of the socket, and uses the private Connect method of the server to bootstrap the rest of the reply process.

The players

The coroutine reads the request line and the MIME headers, and passes that information to the Server's dispatch method

The players

The dispatch method returns a dict

The players

The coroutine then creates an instance of httpd::reply

The players

The players

Simple Serverside Content Example

To generate server side content directly from a url requires modifying the Server object. The Dispatch_Local method is empty by default, and intended to be replaced by the application. If that method returns a non-empty dict, the server ends the dispatch process, and returns that reply to the coroutine, who will in turn pass that to the httpd::reply instance.

Simple Serverside Content Example

# Hijack the dispatcher method on the server
oo::objdefine HTTPD {
  method Dispatch_Local data {
    # Only match up requests for demo
    if {[dict get $data REQUEST_URI] eq "demo"} {
      set reply $data
      dict set reply mixin content ::mydemo
      return $reply
    }
  }
}

Simple Serverside Content Example

Simple Serverside Content Example

clay::define ::mydemo {
  method content {} {
    my puts [my html_header {Hello World!}]
    my puts {<h1>Demo page</h1>}
    my puts "Generated at [clock format [clock seconds]]"
    my puts [my html_footer]
    # NOTE METHOD DOES NOT RETURN A VALUE
  }
}

Simpler Serverside Content Example

HTTPD plugin dispatch ::httpd::plugin.dict_dispatch

HTTPD uri direct * demo {} {
  my puts [my html_header {Hello World!}]
  my puts {<h1>Demo page</h1>}
  my puts "Generated at [clock format [clock seconds]]"
  my puts [my html_footer]
  # NOTE METHOD DOES NOT RETURN A VALUE
}

Simple Serverside Content Example

Simple Serverside Content Example

HTTPD uri direct * rawmime {} {
  my puts [my html_header {Hello World!}]
  set mimetxt [my clay get mimetxt]
  my puts {<h1>Raw Mime</h1>}
  my puts "<pre>$mimetxt</pre>"
  my puts [my html_footer]
}

Simple Serverside Content Example

HTTPD uri direct * texttime {} {
  my reply set Content-Type text/plain
  my puts "This URI was [my request get REQUEST_URI]"
  my puts "Generated at [clock format [clock seconds]]"
}

.tml files

> cat ~/htdocs/demotml.tml

[my html_header {Hello World}]

<h1>Demo page</h1>
Generated at [clock format [clock seconds]]

[my html_footer]

Rest API Example

HTTPD uri direct * restapi {} {
  # Have the reply do its own dispatching
  set URI [split [my request get REQUEST_URI] /]
  lassign [split $URI] base method uuid
  set class [find_class $method]
  if {$class eq {}} {
    # Errors will turned into the appropriate HTTP reply
    tailcall my error 404 {Not Found} {}
  }
  # Mix in a new behavior
  my clay mixinmap rest $class
  my reply set Content-Type application/json
  # Pass off control to a method brought in by the mixin
  tailcall my RestContent
}

Toadhttpd

To run a real web facing server requires a lot of facilities that are beyond the scope of what an embedded web server would provide. At least a web server that is small enough for inclusion in tcllib.

A parallel project, Toadhttpd extends the httpd module in tcllib into fully fledged web server.

Toadhttpd

• Hardened against botnet attacks
• Plug-in architecture
• Bootstrap and jQuery Integration
• Sqlite Based:
  • Dispatch Engine
  • Cacheing Mechanism
  • Logging
  • IP Blocking
• SCGI Proxy Support
• Built-in Fossil Mirroring Support

Q&A

And if you want to play with httpd, (and you have a recent Tcllib installed)

just:

package require httpd