Launch planning

[kytrinyx]

Note: the launch checklist was woefully outdated, since this got created over two years ago, so I deleted it.

If you are interested in leading the charge on getting this track launched, then comment below, and I'll help get this party started based on whatever is current at the time.

We have a launch guide that we'll keep up to date with changes here: https://github.com/exercism/docs/blob/master/language-tracks/launch/README.md

[bdw429s]

Moving our other conversation from #25 over here. That's cool that it's based around tests. So, is it kind of like a Koan? There was a Coldfusion Koans project before https://github.com/nodoherty/ColdFusion-Koans as well as a ColdBox MVC Koans https://github.com/gpickin/Coldfusion-Coldbox-Koans
Both of those projects are pretty old. They were cool, but I'm not sure how many people really used them. The fact that this is test based is also interesting to me since my company also makes the de facto CF testing framework called TestBox (Those koans used an older library called MXUnit, that fell out of maintenance). CommandBox CLI is both a package manager that can install TestBox plus is has TestBox commands integrated. That means, with a couple JSON config files, you could build a repo that when cloned would just need the following CLI commands run:

# install dependencies like TestBox
CommandBox> install
# Start up ad-hoc server in this folder
CommandBox> server start
# Run test suite 
CommandBox> testbox run
# OR watch dir for file changes and re-run tests
CommandBox> testbox watch

That is about as easy as it gets for CFML. The server start command will pop open the browser with HTML test results and the testbox run and testbox watch commands would be for people who preferred to use the CLI. TestBox supports an xUnit style tests as well as BDD style, using closures like Jasmine does. It would be great to maybe figure out the structure and get one test running and then see if I can get some people to contribute in the community.

[kytrinyx]

So, is it kind of like a Koan?

Yeah, the inspiration for this came from a combination of things, Koans was one of them. Also the JavaRanch Cattle Drive (which doesn't have tests, but has volunteers who give feedback on toy problems). Also I was helping mentor some people who were career-switching into programming, and we gave them warm-up exercises every day. At first we didn't give them test suites, but many of them struggled with the basics of breaking problems down, and flailed around a bit so I ended up writing test suites to help guide them and (hopefully) teach them good habits. Then Exercism was automation for the above.

The fact that this is test based is also interesting to me since my company also makes the de facto CF testing framework called TestBox

That is very cool. It does sound like the way to go for this is TestBox + CommandBox CLI.

We don't have people clone a repo, we have a cross-language CLI with a couple of commands to download exercises / upload solutions. I'm in the middle of rewriting the CLI now so that it can have a one-time "setup" for each language that you would run to download anything you need for the exercises (such as the json config that you mentioned).

I think the best first step right now to explore this would be to implement a "hello world" exercise for this track using TestBox and CommandBox CLI and see what would need to go in the installation instructions (in docs/) and what the Exercism CLI would need to download in addition to the exercise.

I've spent my day working on a "how to launch a track" guide... I'm almost at the point where I can push up something useful for you to look at, but my brain is fried so I'm going to need to go walk around for a bit and reread in case I'm accidentally incoherent.

I'll get back to you in this thread in a few hours or so.

[kytrinyx]

Ok, it starts here: https://github.com/exercism/docs/blob/master/language-tracks/launch/README.md

I've tried to balance providing enough context without making it a wall of text. Let me know if this is useful and where things are confusing.

[bdw429s]

Cool, thanks. I'll give that a read. I don't have too much free time, but this sort of promotion is the sort of stuff CF needs so it would be cool to get it started.

I'm in the middle of rewriting the CLI now

What is the CLI written in? I'll note that CommandBox is great for writing interactive CLIs. It has the power of J2EE baked in with out of the box stuff like ANSI formatting, user interactivity, decent native OS execution bridge, and even pretty easy DB access via any JDBC driver :)
https://www.ortussolutions.com/blog/dont-touch-that-node-write-your-own-command-line-tools-in-cfml
</cfshameless_plug>

[bdw429s]

Ok, I read through that link you sent and it all seems pretty straight forward. I like how you're driving all the information in the site off of readme files, JSON data, and directory structures inside the repos. I glanced through what's in the CFML track right now and there's the one leap example. I see whoever created that actually created a TestBox spec, but it doesn't look like they actually put in any plumbing or instructions on how to run the tests. That should be easy enough, though.

I think my biggest question right now is exactly how a user "gets" the CFML track. I understand your CLI will download the files, but where to? Is the user expected to edit stuff directly in that folder? Should there be a test runner set up for each test separately, or just one big test runner that runs all the examples? I'm thinking it would be best if someone cd's into a specific directory for an example and then has everything they need right there to run the tests but I'm already seeing a lot of really redundant code being copied into each folder that doesn't sound smart either. I'm also debating whether we should have the user start up a web server and run the tests there, or just do it 100% from the commandline. The latter isn't something that's very familiar to existing CF devs, but probably pretty common to anyone else. I can create a slick little base task runner that's included from a placeholder tasks in each exercise. Or, I could wrap up a custom Command for Exercism as well but that would involve an extra step of the user installing CommandBox and then installing a module into it. Dunno, maybe it's 6 of one and half a dozen of the other. I'm just trying to think what the smoothest path is that doesn't get off the weeds of tooling or unnecessary boilerplate. Maybe it's a good thing that I expose the users to the plumbing for running a unit test instead of abstracting it all away.

So the second big question is what do I do to get started messing with some of this stuff? Is everything done via pull requests? Do I need to make some changes, submit a pull, wait for you to merge it, then install the CF track via your CLI, and then test it for every change? Is there a faster feedback loop than that?

Thanks!

[bdw429s]

Hi @kytrinyx any feedback on my questions from yesterday? I spent a little time last night playing with a simple CommandBox task runner that would do pure CLI execution of a CFML TestBox test suite from a single command like so (The output makes a bit more sense when you can see the ANSI color formatting):

CommandBox:leap> task run
TestBox v2.5.0+107
---------------------------------------------------------------------------------
| Passed  | Failed  | Errored | Skipped | Time    | Bundles | Suites  | Specs   |
---------------------------------------------------------------------------------
| 6       | 0       | 0       | 0       | 6 ms    | 1       | 1       | 6       |
---------------------------------------------------------------------------------
=================================================================================
C__Users_Brad_development_Documents_GitHub_coldfusion_exercises_leap.leap_tests (2 ms) [Suites/Specs: 1/6]
[Passed: 6] [Failed: 0] [Errors: 0] [Skipped: 0]
---------------------------------------------------------------------------------
+Leap Tests
    testNonLeapEvenYear (0 ms)
    testNonLeapYear (1 ms)
    testLeapYear (0 ms)
    testY2K (0 ms)
    testCentury (0 ms)
    testfourthCentury (0 ms)
---------------------------------------------------------------------------------
Passed || Skipped || Exception/Error || Failure

Turns out the entire task runner for this is only 15 lines of code, so I guess it's not too bad if each exercise has a small bit of boilerplate. That still doesn't preclude the option to include this logic from a higher directory. I'm also still debating whether to set up a web based running for people who are more comfortable starting a server and hitting their tests via an HTML report in the browser. I like the options, but I also don't want to be confusing with a bunch of different ways to do it.

[bdw429s]

Ok @kytrinyx , here's my suggested format. I reworked the existing "Leap" exercise and shared it in the form of this pull request:
#26

Please provide your feedback.

[kytrinyx]

What is the CLI written in?

It's written in Go. It lets us distribute a binary with no runtime requirements. We don't need any database or anything (it's all just local filesystem stuff and HTTP calls to talk to the API). It's working well for us.

any feedback on my questions from yesterday?

Sorry—it's been a rough week. I'm only just seeing this. I'll respond below.

I think my biggest question right now is exactly how a user "gets" the CFML track.

Right. Take a look at the homepage of the v2 site https://v2.exercism.io (scroll down, it explains the basic interaction). Then take a look at the Go track page: https://v2.exercism.io/tracks/go

People join a track, which gives them a web dashboard showing their progress. Then they're guided through installing the CLI and configuring it. Then they download the first exercise, which will go to wherever on their computer they want these exercises. For the sake of argument, let's say their Exercism workspace is ~/exercism, then the leap exercise would be downloaded to ~/exercism/coldfusion/leap, and they would solve the exercise right there in the directory.

When they're done, they would upload their solution file to the site.

Should there be a test runner set up for each test separately, or just one big test runner that runs all the examples? I'm thinking it would be best if someone cd's into a specific directory for an example and then has everything they need right there to run the tests but I'm already seeing a lot of really redundant code being copied into each folder that doesn't sound smart either.

Yeah, they should be able to run each test individually. If we need to download some boilerplate just once, then we can do that. There's a setup step that we run for each track, and we can specify files to download just once. For now, just assume that they'll have nothing but the leap directory, and that they need to make the tests run... what would be missing in that case?

I'm also debating whether we should have the user start up a web server and run the tests there, or just do it 100% from the commandline. The latter isn't something that's very familiar to existing CF devs, but probably pretty common to anyone else.

That's a great question. I would lean towards command-line, but frankly if that's not common in the CF world, then I would prefer giving people the basics so they're familiar with what's actually used. In this case it sounds like a little web server would be a good call.

Maybe it's a good thing that I expose the users to the plumbing for running a unit test instead of abstracting it all away.

Yes! I think it's pretty important for people to understand the basics, and plumbing is pretty important in this case.

So the second big question is what do I do to get started messing with some of this stuff? Is everything done via pull requests? Do I need to make some changes, submit a pull, wait for you to merge it, then install the CF track via your CLI, and then test it for every change? Is there a faster feedback loop than that?

Yes and no. I prefer everything to be done via pull requests, but I'll add you as a maintainer to the track so you can merge your own PRs. Using pull requests makes it easier for people to see what's going on in the track.

Ideally for the track maintainers and contributors we should have a test runner that lets us run all of the tests in the track against a reference solution.

Turns out the entire task runner for this is only 15 lines of code, so I guess it's not too bad if each exercise has a small bit of boilerplate.

That's great! I agree. A bit of boilerplate is totally fine.

I'm also still debating whether to set up a web based running for people who are more comfortable starting a server and hitting their tests via an HTML report in the browser. I like the options, but I also don't want to be confusing with a bunch of different ways to do it.

Yeah, I hear you. The command-line option feels really nice and light-weight, but again: I'm heavily biased towards command-line, as I live much of my life there.

I reworked the existing "Leap" exercise and shared it in the form of this pull request:

Sweet, I'll take a look now.

[ilya-khadykin]

@bdw429s, I suggest adding necessary user facing docs (docs dir) as a next step (since you've basically done with #26):

• INSTALLATION.md - how to install the language and any dependencies
• TESTS.md - how to run the tests
• ABOUT.md - short, enticing introduction to the language
• LEARNING.md - where to get an introduction to the language if you're learning it for the first time
• RESOURCES.md - references, tools, etc.

More info on that: https://github.com/exercism/docs/blob/master/language-tracks/documentation/for-consumers.md

It's also a good idea to add CONTRIBUTING.md in root of the repo to describe how one can contribute to the track - https://github.com/exercism/docs/blob/master/language-tracks/documentation/for-contributors.md

And you can try to generate exercise README.md with configlet generate as described in #26 (comment)

[bdw429s]

Thanks @m-a-ge I'll branch off of my first branch and start stubbing those out. It will be important to get all this documented!

[bdw429s]

@m-a-ge I have started a new pull that includes all the docs you listed and a few more. Any feedback is welcome.

#27

[bdw429s]

I have created another pull which introduces an exercise template to make it very easy for other contributors to create exercises: #28
This comes with a CLI task runner (written in CFML, of course!) that will allow future contributors to stub out all the files they need with the correct names and text replacements via a single command.

box task run tasks/ScaffoldExercise

[bdw429s]

10 more pulls added bringing us to 12 total exercises:

#39
#38
#37
#36
#35
#34
#33
#32
#31
#30

What else needs to happen to launch the ColdFusion track?

[bdw429s]

7 more exercise pulls, bringing us to 19 right now

#40
#41
#42
#43
#44
#45
#46

[bdw429s]

5 more exercises:

#47
#48
#49
#50
#51

[bdw429s]

Hi @kytrinyx , I hope you were able to relax during your time away. Let me know when you're available to go over the outstanding pulls for the ColdFusion track.

[bdw429s]

@kytrinyx So I assume I'm clear to start merging stuff. Here's a question. Now that I have merge permissions, is the typical workflow to create branches inside of the exercism/coldfusion repo and then merge from there, or to continue creating branches in my own fork and merge from there?

[bdw429s]

@kytrinyx All pulls are merged. Can you give me a thumbs up for toggling the active flag?

Also, is there a manual process in place for the latest exercises to show up here or do I just need to be patient and wait for some jobs to run?

[bdw429s]

Hi @kytrinyx I'm still looking for some confirmation of how the latest ColdFusion exercises will show up on the exercism site. I've merged all the pulls and switched the active flag in the config.json file but http://exercism.io/languages/coldfusion/exercises is still showing the old content. Is there something else that needs done? Thx.

[kytrinyx]

@bdw429s—I'm slowly catching up from my week out.

The exercises should show up automatically, provided that I deploy... I think I've forgotten to deploy in the past three days (work has been very intense). I'm deploying now.

In v2 we'll need to run a specific command to add the track to the website, but any subsequent changes to the track will be reflected on the site automatically, without a deploy.

Can you give me a thumbs up for toggling the active flag?

Sorry. Yes! I'm glad you went ahead and activated the track.