Simplifying scala scripts : Adding #include support to your scripts

Test a #include support example by executing the following 5 lines :

$ wget http://dnld.crosson.org/bootstrap.tar.gz
$ tar xvfz bootstrap.tar.gz
$ cd bootstrap
$ sbt assembly
$ ./scripts/test.scala

The test.scala example script is the following :

#!/bin/sh
DIRNAME=`dirname "$0"`
exec java -jar "$DIRNAME"/bootstrap.jar "$0" "$@"
!#

#include "shell.scala"

cd("/etc/")

"ls" #| "grep net" !

go to /etc directory, and prints files which contain net keyword in their names.

test.scala script includes the following file : scripts/include/shell.scala

import sys.process.Process
import sys.process.ProcessBuilder._
 
case class CurDir(cwd:java.io.File)
implicit def stringToCurDir(d:String) = CurDir(new java.io.File(d))
implicit def stringToProcess(cmd: String)(implicit curDir:CurDir) = Process(cmd, curDir.cwd)
implicit def stringSeqToProcess(cmd:Seq[String])(implicit curDir:CurDir) = Process(cmd, curDir.cwd)

implicit var cwd:CurDir=scala.util.Properties.userDir
def cd(dir:String=util.Properties.userDir) = cwd=dir

This file contains some definitions to make possible for the user to change current directory.

All the include mechanism logic in defined as follow :

package fr.janalyse.script

import scala.tools.nsc.ScriptRunner
import scala.tools.nsc.GenericRunnerCommand
import scala.io.Source
import java.io.File

object Bootstrap {
  val defaultOptions = List("-nocompdaemon","-usejavacp","-savecompiled", "-deprecation")
  val defaultExpandedScriptExt = ".pscala"
  
  val includeRE = """\s*#include\s+"(.+)"\s*"""r
  
  def expand(file:File, availableIncludes:List[File]) : List[String] = {
    val content=Source.fromFile(file).getLines().toList
    // First we remove "shell" startup lines, everything between #! and !#
    val cleanedContent = content.indexWhere { _.trim.startsWith("!#") } match {
      case -1 => content
      case i  => content.drop(i+1)
    }
   // Then we expand #include directives
   cleanedContent flatMap {
     case includeRE(filename) =>
       val fileOpt = availableIncludes find {_.getName() == filename}
       fileOpt orElse {
          throw new RuntimeException("%s : Couln't find include file '%s' ".format(file.getName, filename))
       }
       fileOpt map { file => expand(file, availableIncludes)} getOrElse List.empty[String]  
     case line => line::Nil
   }
  }
    
  def main(cmdargs:Array[String]) {
    val command = new GenericRunnerCommand(defaultOptions ++ cmdargs.toList)
    val scriptDir = new File(cmdargs(0)).getParentFile()
    val includePath = List(new File(scriptDir, "include"), scriptDir)
    val availableIncludes = includePath filter {_.exists()} flatMap {_.listFiles()}
    val scriptname = command.thingToRun 
    val script = new File(scriptname)
    val richerScript = new File(scriptname.replaceFirst(".scala", defaultExpandedScriptExt))
    
    if (script.exists()) {
      val jars = util.Properties.javaClassPath.split(File.pathSeparator) map {new File(_)} collect {
        case f if (f.exists() && f.isFile()) => f 
      }
      val jarsLastModified = (jars map {_.lastModified()} max)
      
      if (!richerScript.exists ||  // -- nothing already available
          (jarsLastModified > richerScript.lastModified) ||   // -- Bootstrap jar is newer
          (script.lastModified > richerScript.lastModified)) { // -- Script has been modified
        val newcontent =  expand(script, availableIncludes).mkString("\n")
        new java.io.FileOutputStream(richerScript) {
          write(newcontent.getBytes())
        }.close()
      }
    }
    ScriptRunner.runScript(command.settings, richerScript.getPath, command.arguments)
  }
}

How does it work :
The principle is to override scala standard script startup mechanism by introducing an additionnal step which consist to expand the script with all includes it contains, and then gives to scala the new script resulting of expansion process.
test.scala becomes test.pscala which will generate the savedcompile file test.pscala.jar. No recompilation will be required as soon as no change occured on test.scala or bootstrap.jar file.

You should also notice that the script is started using 'exec java -jar "$DIRNAME"/bootstrap.jar "$0" "$@"' and not 'exec scala ...' because bootstrap is an assembly jar which contains everything to run and compile scala scripts, and even more if you want, as it can include any third parties you may need, just add library dependencies ! So you only need one file, bootstrap.jar, to run any scala scripts, nothing to install, just one file to upload.

SBT build configuration : bootstrap/build.sbt

import AssemblyKeys._

seq(assemblySettings: _*)

name := "bootstrap"

version := "0.1"

scalaVersion := "2.9.1"

libraryDependencies <++=  scalaVersion { sv =>
   ("org.scala-lang" % "scala-swing" % sv) ::
   ("org.scala-lang" % "jline"           % sv  % "compile") ::
   ("org.scala-lang" % "scala-compiler"  % sv  % "compile") ::
   ("org.scala-lang" % "scala-dbc"       % sv  % "compile") ::
   ("org.scala-lang" % "scalap"          % sv  % "compile") ::
   ("org.scala-lang" % "scala-swing"     % sv  % "compile") ::Nil   
}

mainClass in assembly := Some("fr.janalyse.script.Bootstrap")

jarName in assembly := "bootstrap.jar"

SBT Plugins configuration : bootstrap/project/plugins.sbt file

resolvers += Classpaths.typesafeResolver

addSbtPlugin("com.typesafe.sbteclipse" % "sbteclipse-plugin" % "2.0.0-M3")

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.7.2")