Hi, As input for designing new features that better support scripting in Pharo, we would benefit from a bit more input around the role scripts play in your development life. To this end, I would like to invite you to share your scripting experience. Here are some questions to start from, but feel free to diverge: - What kind of actions do you put in scripts? - Do you rely on script files stored on the disk at all? - Where do you store your scripts (in one folder, in many folders)? - Do you version your scripts? - Do you store them in separate files with dedicated names, or tend to put multiple snippets in one larger file? - Do you rely on the names of the scripts? - Why do you use a script file and not a class? Cheers, Doru -- www.tudorgirba.com "Every thing has its own flow"
On Fri, Dec 12, 2014 at 12:14 PM, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
As input for designing new features that better support scripting in Pharo, we would benefit from a bit more input around the role scripts play in your development life.
To this end, I would like to invite you to share your scripting experience. Here are some questions to start from, but feel free to diverge:
- What kind of actions do you put in scripts?
Definition of configuration of system (e.g. the tree of elements that are loaded by the generic software for a given customer. The file is written by the field engineer using the DSL classes and method). Definition of parameters (e.g. Seaside ports, email servers, ...) Using Sebastian Sastre's ConfigurationFiles. Patches for methods in a patches/ folder Startup script, cleaning caches, reloading structures, overriding parameters, starting RFB, loading new Mczs, ...
- Do you rely on script files stored on the disk at all?
Yes, in my conf/ and patches/ for example.
- Where do you store your scripts (in one folder, in many folders)?
Some folders, including one I didn't mentioned here, which is set of files to help me with some aspects that are coming again and again (e.g. configure GT tools, put back normal tools etc). Some things I keep around: https://github.com/philippeback/scripts/tree/master/pharo/Various Also as gists https://gist.github.com/philippeback
- Do you version your scripts?
Yes https://github.com/philippeback/scripts/tree/master/pharo and client code has a branch for each client deployment site.
- Do you store them in separate files with dedicated names, or tend to put multiple snippets in one larger file?
Depends on what I do.
- Do you rely on the names of the scripts?
Yes, for configuration files.
- Why do you use a script file and not a class?
There are two things. I do have utility classes with a lot of methods acting as a facade to the system, so that one can use it in a workspace easily. Now, these are used in a script to do things like: AcmeSystem stop; resetAllCaches; reloadConfiguration; ... start. These would be in a AcmeSystemStart.st that gets loaded by the headless image on the server with the st command line handler. Also useful is that I do have a command line handler that is wrapped with a acmetools.sh script that calls into the AcmeSystem class for administrative tasks, like in: ./acmetools.sh --list-users ./acmetools.sh --createUser xyz --... HTH Phil
Cheers, Doru
-- www.tudorgirba.com
"Every thing has its own flow"
-- --- Philippe Back Visible Performance Improvements Mob: +32(0) 478 650 140 | Fax: +32 (0) 70 408 027 Mail:phil@highoctane.be | Web: http://philippeback.eu Blog: http://philippeback.be | Twitter: @philippeback Youtube: http://www.youtube.com/user/philippeback/videos High Octane SPRL rue cour Boisacq 101 | 1301 Bierges | Belgium Pharo Consortium Member - http://consortium.pharo.org/ Featured on the Software Process and Measurement Cast - http://spamcast.libsyn.com Sparx Systems Enterprise Architect and Ability Engineering EADocX Value Added Reseller
Hi 2014-12-12 8:14 GMT-03:00 Tudor Girba <tudor@tudorgirba.com>:
Hi,
As input for designing new features that better support scripting in Pharo, we would benefit from a bit more input around the role scripts play in your development life.
To this end, I would like to invite you to share your scripting experience. Here are some questions to start from, but feel free to diverge:
- What kind of actions do you put in scripts?
Temporary workflows.
- Do you rely on script files stored on the disk at all?
Sometimes yes.
- Where do you store your scripts (in one folder, in many folders)?
I use the ScriptManager to store my scripts. When using files I store them in many folders (per image/project)
- Do you version your scripts?
Yes
- Do you store them in separate files with dedicated names, or tend to put multiple snippets in one larger file?
- I mostly rely in ScriptManager for this but sometimes I put one script per tab in Notepad++ (Notepad++ doesn't force to write a name and doesn't remove tabs if you don't save). - If multiple snippets are located in one file then I loose syntax highlighting.
- Do you rely on the names of the scripts?
Yes, to identify the script task. But never to import/require by name.
- Why do you use a script file and not a class?
Because conceptually are different. A class represents a conceptual evolution of what could be initially a script. And because we do not have yet a Workflow system like Galaxy or myExperiment :) Cheers, Hernán
Cheers, Doru
-- www.tudorgirba.com
"Every thing has its own flow"
Hi Tudor, I'll reply you below. El Fri Dec 12 2014 at 8:15:40 AM, Tudor Girba <tudor@tudorgirba.com> escribió:
- What kind of actions do you put in scripts?
- Configuration expressions (to startup a headless image). - One shot tasks run by cron jobs. - Patching expressions evaluated by CodeImporter
- Do you rely on script files stored on the disk at all?
Absolutely.
- Where do you store your scripts (in one folder, in many folders)?
Setup and configuration scripts in the image directory. Cron task in a jobs/ subdirectory. Updates/Patches in updates/
- Do you version your scripts?
I do. Using git along the static assets.
- Do you store them in separate files with dedicated names, or tend to put multiple snippets in one larger file?
Normally it's only one setup file. In the case of cron tasks They can contain parts of the setup and a message send that starts all the actions. - Do you rely on the names of the scripts?
Just for reference. And in the case of cron tasks the file name is static.
- Why do you use a script file and not a class?
I can modify most of the tasks by using a text editor (vim) through ssh. Regards!
Esteban, Interesting that cron thing with jobs/. Maybe could we list these patterns as they seem to be pretty generic for those business apps of ours. Phil
On Dec 12, 2014, at 9:14 AM, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
As input for designing new features that better support scripting in Pharo, we would benefit from a bit more input around the role scripts play in your development life.
To this end, I would like to invite you to share your scripting experience. Here are some questions to start from, but feel free to diverge:
- What kind of actions do you put in scripts?
Automation. Mostly devops and orchestration (so they make Jenkins life easier (not true but it keeps the CI jobs more maintainable)) Truth is that developers save time too
- Do you rely on script files stored on the disk at all?
every day
- Where do you store your scripts (in one folder, in many folders)?
depends on the project In flow there are some scripts in some folders that creates a convenient syntax and we keep some of them in a hidden folder .hood
- Do you version your scripts?
git does it for me
- Do you store them in separate files with dedicated names, or tend to put multiple snippets in one larger file?
Iâm not considering snippets and scripts in the same category. For snippets I use workspaces. Typically I use a class with the name of the app and in its class side I put convenient methods. Some examples: Airflowing start. âstarts all the services used inside the image" Airflowing start. âstops them" Airflowing restart. âstops them, do 5 garbagecollects and starts" Airflowing maintainer update executeOn: âdatabaseNameâ. âexecutes the current update (a custom class with custom actions) on 1 particular database" Airflowing maintainer update execute. "executes the current update in all databasesâ There are others for many day-to-day things, including capturing monthly payments PetShop <https://github.com/flow-stack/petshop/tree/develop>âs backend uses that idea too PetShop stop; reset; start. PetShop <https://github.com/flow-stack/petshop/tree/develop>âs frontend is (is flow <https://github.com/flow-stack/flow>-based so weâre talking about Amber <http://amber-lang.net/>) using that idea too PetShop start âregisters the app, publishes shared remote objects and instantiates the main controller) Same for many other private projects
- Do you rely on the names of the scripts?
Not only script names but also folder names. This is what makes the terminal to be more productive. More airflowing examples: $ <â youâre in the root of the project in dev $ ./actions <â type âa' and pres tab once and you get the actions folder, press tab again and it lists all of the things you can do, some of them are: getDataToSpace getRelease pullDev freshDev getFlowingWorker getStagingImage pullMaster when you do that, as you probably guessed, the script naming criteria becomes pretty close to method naming criteria Another relaying of the names of the scripts is for keeping configuration out of the image as used by ConfigurationFiles <http://smalltalkhub.com/#!/~sebastianconcept/ConfigurationFiles>: Itâs a very simple class that make a bunch of assumptions to allow you to have all the configuration files you want under conf/ directory so you can inside the image do things like: ConfigurationFiles at: #Zinc at: #port ConfigurationFiles at: #MongoDB at: #databaseName ConfigurationFiles at: #VNC at: #port etc etc And here is an example of how the VNC file looks like $ cat {imageDir}/conf/VNC.conf "############ VNC accessor. Settings for the VNC server so headless images can provide an UI through VNC clients" Dictionary new at: #use put: true; at: #localOnly put: true; at: #port put: 5908; at: #interactivePassword put: âsecret'; yourself
- Why do you use a script file and not a class?
As mentioned, I do use classes too. Mostly to do actions inside the worker images. Also to build images. The next examples use the ./load or ./build or ./cleanBuild script to create the development setup of the respective projects https://github.com/sebastianconcept/Mapless/blob/master/cleanBuild <https://github.com/sebastianconcept/Mapless/blob/master/cleanBuild> and to implement the image customization there is the {ProjectName}Builder class that those scripts file in to get that done. Here is Mapless example: https://github.com/sebastianconcept/Mapless/blob/master/MaplessBuilder.st <https://github.com/sebastianconcept/Mapless/blob/master/MaplessBuilder.st> Here is same idea used for the flowâs backend: https://github.com/flow-stack/flow/blob/master/backend/FlowBuilder.st <https://github.com/flow-stack/flow/blob/master/backend/FlowBuilder.st> BTW in flowâs root folder there is ./cleanBuild to trigger building both, back and frontend: https://github.com/flow-stack/flow/blob/master/cleanBuild <https://github.com/flow-stack/flow/blob/master/cleanBuild> here is how it cleans and triggers the build https://github.com/flow-stack/flow/blob/master/cleanBuildFrontend <https://github.com/flow-stack/flow/blob/master/cleanBuildFrontend> the frontendâs concrete implementation: https://github.com/flow-stack/flow/blob/master/frontend/build <https://github.com/flow-stack/flow/blob/master/frontend/build> Here is how it ./startAll hides a multiplatform start https://github.com/flow-stack/flow/blob/master/startAll <https://github.com/flow-stack/flow/blob/master/startAll> and here is the OS X implementation that opens one terminal per service started: https://github.com/flow-stack/flow/blob/master/.hood/platform/osx/basicStart... <https://github.com/flow-stack/flow/blob/master/.hood/platform/osx/basicStart...>
Cheers, Doru
-- www.tudorgirba.com <http://www.tudorgirba.com/>
"Every thing has its own flow"
Hi,
As input for designing new features that better support scripting in Pharo, we would benefit from a bit more input around the role scripts play in your development life.
To this end, I would like to invite you to share your scripting experience. Here are some questions to start from, but feel free to diverge:
- What kind of actions do you put in scripts? some expressions - Do you rely on script files stored on the disk at all? No Just class
- Where do you store your scripts (in one folder, in many folders)? No never - Do you version your scripts?
No or a class methods
- Do you store them in separate files with dedicated names, or tend to put multiple snippets in one larger file?
No
- Do you rely on the names of the scripts? No
- Why do you use a script file and not a class? I don't
Cheers, Doru
-- www.tudorgirba.com <http://www.tudorgirba.com>
"Every thing has its own flow"
Hi Tudor, On Fri, Dec 12, 2014 at 3:14 AM, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
As input for designing new features that better support scripting in Pharo, we would benefit from a bit more input around the role scripts play in your development life.
To this end, I would like to invite you to share your scripting experience. Here are some questions to start from, but feel free to diverge:
- What kind of actions do you put in scripts?
Actions that build and/o bootstrap systems. These are typically of two different kinds: - shell scripts to launch images invoked with Smalltalk scripts - Smalltalk scripts - Do you rely on script files stored on the disk at all?
Yes. - Where do you store your scripts (in one folder, in many folders)?
Typically each project that uses one or more scripts has those scripts in the main directory or a scripts subdirectory of that project's main directory. - Do you version your scripts?
The version control system for the project does that. - Do you store them in separate files with dedicated names, or tend to put
multiple snippets in one larger file?
Definitely lots of small scripts with separate names. Some times they call each other. - Do you rely on the names of the scripts?
Yes. They typically indicate what the script does, e.g. buildspurtrunkimage.sh which in turn uses UpdateSqueakTrunkImage.st BuildSpurTrunkImage.st & LoadSpurPackages.st
- Why do you use a script file and not a class?
Ah, sometimes the script file contains a class. For example, the Newspeak bootstrap. But that class is eliminated before the image is finally saved. The choice as to whether to use a class or not depends on the complexity of the task, the usefulness of breaking things into methods etc. Sometimes the script contains many lines, sometimes it's jut way too simple for a class, e.g. BuildSqueakTrunkVMMakerImage.st: "Change the update stream to trunk, update, then build as per 4.5" [MCMcmUpdater defaultUpdateURL: 'http://source.squeak.org/trunk'; updateFromServer] valueSupplyingAnswer: true. "don't use fileIn; it causes an error when the saved image is started" Compiler evaluate: (FileStream oldFileNamed: 'BuildSqueak45VMMakerImage.st') contentsOfEntireFile The advantage that a non-class single doit script has over a class is that the former is much easier to edit with any text editor. For a class one really needs the IDE, and for some tasks one wants to edit outside of the IDE. If you want to look at some examples of scripts I use see - the shell scripts named foo.sh and the Smalltalk scripts named Foo.st, which together build Spur, in http://www.squeakvm.org/svn/squeak/branches/Cog/image - the shell scripts named foo.sh and the Smalltalk scripts named Foo.st, which together build Newspeak, in https://bitbucket.org/newspeaklanguage/nsboot_bleeding_edge/src Cheers,
Doru
-- www.tudorgirba.com
"Every thing has its own flow"
-- HTH, Eliot
On 12 Dec 2014, at 12:14, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
As input for designing new features that better support scripting in Pharo, we would benefit from a bit more input around the role scripts play in your development life.
To this end, I would like to invite you to share your scripting experience. Here are some questions to start from, but feel free to diverge:
Every image that I am working in for some time contains multiple open workspaces that I need and depend on, that I dare not close because they might contain important stuff.
- What kind of actions do you put in scripts?
Starting points for coding, exploring, starting/stopping services, loading code, generally useful expressions, snippets of code that will evolve in something else later on.
- Do you rely on script files stored on the disk at all?
Not really, but I tried (both using save/load and ScriptManager) - I think the extra step is too much, a permanent workspace in an image is so much easier.
- Where do you store your scripts (in one folder, in many folders)?
Just next to the image, mostly. But I tried structured saving with ScriptManager.
- Do you version your scripts?
No
- Do you store them in separate files with dedicated names, or tend to put multiple snippets in one larger file?
Both
- Do you rely on the names of the scripts?
If there are names, yes.
- Why do you use a script file and not a class?
Some tools end up as class methods, if not real code. But changing a class (code) is more serious than playing/experimenting/exploring code in a workspace. === One of the things that I do is mail snippets around internally, so that I and others have them in their mailbox for later reference. === We need something very low friction, like automatic saving/remembering.
Cheers, Doru
-- www.tudorgirba.com
"Every thing has its own flow"
Am 12.12.2014 um 12:14 schrieb Tudor Girba <tudor@tudorgirba.com>:
Hi,
As input for designing new features that better support scripting in Pharo, we would benefit from a bit more input around the role scripts play in your development life.
To this end, I would like to invite you to share your scripting experience. Here are some questions to start from, but feel free to diverge:
For me there are two kind of environments that are imported. On the one hand there is the desktop with a GUI, something that I start manually and that is being saved regularly for me to stop and continue working. On the other hand it is a server environment where everything needs to be started automatically without any GUI. This environment is never saved and always started from the same saved state.
- What kind of actions do you put in scripts?
Setup, configuration and orchestration of components. Short snippets for code I don't want to put in methods, snippets for resetting things etc. Examples are: starting zinc server, adding and configure persistence layer to a stack, resetting model state etc.
- Do you rely on script files stored on the disk at all?
On the desktop I store workspaces to files as backup and to be able to move them to a freshly build image. On the server the whole setup is done with scripts
- Where do you store your scripts (in one folder, in many folders)?
On the desktop I store them in the working directory of the image. I have separate directories for all of my projects. So the project dependent workspace files are just in that project folder. I'd like the idea of organizing scripts but then accessing/navigating to them is more effort then to type it again. I also store scripts in text documents for e.g. having quick access to Gofer snippets in order to load common modules I use. This is mainly to set up the project before writing a metacello config. And not all of those components can be accessed via configuration browser. On the server I've built a utility that accesses a directory full of snippets in a Unix SysV style manner. As an example for our mobility server: root@2d-apis:/opt# ls -1 mobility-server/launch.d/ 10_restore-ui-manager 20_rfb-vnc-server 30_logging 40_mongodb 50_start-mobility 60_scheduler 95_server-setup The prefix number organizes the order of execution of the snippets. The snippets are a mixture of shared and individual scripts. Meaning scripts like restore ui manager and rfb-vnc-server are shared among all installations because they are always the same. I just symlink them into my projects launch.d. A script like rfb-vnc-server looks like this root@2d-apis:/opt# cat mobility-server/launch.d/20_rfb-vnc-server self optionAt: 'vncport' ifPresent: [:portString| RFBServer reset. rfb := RFBServer current. self optionAt: 'vncpassword' ifPresent: [:password| rfb setFullPassword: password ]. rfb start: portString asNumber. self log: 'RFB vnc server started on port ', portString ]. This way I can switch rfb on and off using a commandline switch that is added to a project configuration. This all is part of image-launcher I want to release for several years now :) If anyone is interested please ask.
- Do you version your scripts?
Workspace scripts are versioned via timemachine backup only. Server side scripts are put into git.
- Do you store them in separate files with dedicated names, or tend to put multiple snippets in one larger file?
On the desktop it is more the latter because everything a project depends is in one or two workspaces. On the server as you could see above everything is separated into indvidual scripts
- Do you rely on the names of the scripts?
On the server for tweaking the order of execution.
- Why do you use a script file and not a class?
Some of the scripts I put into classes when they grow up to be an utility function. Parts that contain contextual information I don't put into classes because I don't want to have filesystem paths, ip addresses and such inside my code and they are too static if something changes. I think putting stuff in classes is not always an option. To me scripts act as glue between the image and outer world where to image have to live in. Scripts are flexible and adhoc. In my case there is a workflow where a jenkins server builds the deployable image. If the script code (e.g. for startup) in the image needs to be tweaked you need to change the image in order to fix it and there are workflows where this would be pain (rebuilding the image and deploy it to have the change in effect). So scripts in text files have superpowers here. Norbert
participants (9)
-
Eliot Miranda -
Esteban A. Maringolo -
Hernán Morales Durand -
Norbert Hartl -
phil＠highoctane.be -
Sebastian Sastre -
stepharo -
Sven Van Caekenberghe -
Tudor Girba