Jobs can now be configured with serial_groups, which can be used to\nensure multiple jobs do not run their builds concurrently. See\n[missing] for more information.
Jobs can now be paused. This prevents newly created builds from\nrunning until the job is unpaused.
To pause a job, go to its page, which is now accessible by clicking\n the job name when viewing a build, and click the pause button next to\n its name in the header.
\n\nThe abort button now aborts asynchronously, and also works when\naborting one-off builds.
If multiple template variables are not bound when configuring via a\npipeline template, all of their names are printed in the error, rather\nthan just the last one.
Resource source and params can now contain arbitrarily nested\nconfiguration.
We’ve upgraded D3, which now does smoother zooming when\ndouble-clicking or double-tapping the pipeline view.
The ’started’ indicator in the legend now does its little dance once\nagain.
Don’t worry, this one’s backwards-compatible.
\n\nIf you’re reading this and haven’t upgraded to v0.44.0 yet, be sure to\nread that guy’s release notes. It’s a doozy.
\n\nThere is an icon at the top right (in the nav bar) that will take you\n to a page listing all builds that have ever run, including one-off\n builds, with the most recent up top.
\n\nResources can now be paused, meaning no new versions will be\ncollected or used in jobs until it is unpaused. This is useful to cut\noff broken upstream dependencies.
Pipeline configurations can now be parameterized via\nfly configure. This allows pipeline configurations to be reused,\nor published with the credentials extracted into a separate (private)\nfile.
The Time resource can now be configured to trigger once, or on an\ninterval, within a time period. This can be used to e.g. run a build\nthat cleans up development environments every night, while no one’s at\nwork.
The super verbose and ugly perl warnings while cloning git\nrepositories has been fixed!
Some pipeline UI quirks have been fixed. Right-clicking no longer\ntriggers dragging around, and the zooming has been bounded (no more\nlosing your pipeline!).
This release is hella backwards incompatible. Read carefully, and ask in\nIRC (#concourse) if you need help!
\n\nWe won’t be making such drastic changes after 1.0, but as long as we’re\nstill figuring things out, we don’t want to collect tech debt or land on\nthe wrong set of primitives.
\n\nPreviously there was basically a working directory that would be\n streamed from step to step, and aggregate steps were relied on to\n place things under subdirectories, which is how inputs to tasks were\n satisfied.
Now, as a plan executes, each step’s produced artifact (for example a\n get step’s fetched bits or the result of a task’s execution) are\n stored in a pool, with the source named after the step.
This change affects many things, but the primary things you’ll notice\n are as follows:
\n\nWhen executing a task step, its inputs are collected from the\npool, rather than blindly streamed from the previous step. This\nmeans aggregate is no longer required to satisfy task inputs, and\ncan now be removed if it’s only wrapping one step.
Tasks are now required to list their set of inputs, otherwise no\ninputs will be streamed in. This is backwards-incompatible, but has\nmany advantages: it’s more explicit, more efficient, and makes it\nclearer where the dependent inputs will be placed in a task’s\nworking directory when it runs.
\n\nWhen a task completes, its resulting working directory is added to\nthe pool, named after the task itself. This is how you would put\nusing artifacts generated by tasks.
The file attribute of a task step must now qualify the path with\nthe name of the source providing the file.
When executing a put step, all sources are fetched from the\npool. Later on we may introduce a change so that put steps declare\ntheir dependencies, but for now streaming everything in is the\nsimplest path forward.
The net effect of this is that any params referring to files in\nput steps must now qualify the path with the source name, as\nthey’re all fetched into subdirectories.
Now that there’s a flat pool of sources, later steps in a build plan\ncan now refer back to previously fetched (or generated) sources,\nrather than having to fetch them again.
So, if before you had a plan that looked like this:
\n\nplan: \n - aggregate: \n - get: something \n - task: generate-foo\n file: build.yml \n - put: foo-bucket \n params: \n from: foo \n...it would now look like this:
\n\nplan: \n - get: something \n - task: generate-foo \n file: something/build.yml\n - put: foo-bucket \n params: \n from: generate-foo/foo \nNotably, the redundant aggregate is gone, the file attribute of\n the task step qualifies the filename with the name of the source\n containing it, and the put step qualifies the path to foo with the\n name of the task that it came from.
Also, the something/build.yml task would now explicitly list its\n inputs, if it wasn’t before. So that could mean changing:
platform: linux \n\n image: docker:///busybox \n\n run: \n path: something/some-script\n...to...
\n\nplatform: linux \n\n image: docker:///busybox \n\n inputs: \n - name: something \n\n run: \n path: something/some-script\nThis has the advantage of making the task config more\n self-documenting, and removes any doubt as to what inputs will be\n placed where when the task starts.
\n\nNote that listing inputs in the task config is not new, and if you\n were already listing them before the semantics hasn’t changed. The\n only difference is that they’re now required.
\n\nTo upgrade, you’ll have to change your manifest a bit:
\n\nOn your workers, replace the gate job with groundcrew and remove\nthe gate properties.
The new tsa job template will have to be added somewhere, and\nconfigured with the atc credentials (the same way gate used to\nbe configured).
Colocating tsa with the atc works out nicely, so that you can\nregister its listening port 2222 with your routing layer (e.g.\nELB), which will already be pointing at the ATC.
To compare, see the example AWS VPC manifest.
\n\nThe main upshot of this change is it’s much easier to securely\n register an external worker with Concourse. This new model only needs\n the worker to be able to reach the ATC rather than the other way\n around.
\n\nBackwards-incompatible: Consul services are now automatically\nregistered based on the jobs being colocated with the agent. For this\nto work, you must edit your deployment manifest and move the\nconsul-agent job to the top of each job template list, and remove\nyour existing Consul services configuration from your manifest.
The get and put steps from a build’s execution can now be hijacked\nafter they’ve finished or errored. Previously they would be reaped\nimmediately; now they stick around for 5 minutes afterwards (same\nsemantics as tasks).
The S3 resource now\ndefaults to the us-east-1 region.
The S3 resource no longer\nfails to check when the configured bucket is empty.
A new BOSH Deployment resource has been introduced. It can be used to\ndeploy a given set of release/stemcell tarballs with a manifest to a\nstatically configured BOSH target. The precise versions of the\nreleases and stemcells are overridden in the manifest before deploying\nto ensure it’s not just always rolling forward to latest.
Two new resources: bosh.io release and bosh.io stemcell, for consuming\npublic BOSH releases and stemcells in a more convenient way.
The event stream is now GZip compressed, which should speed up build\nlogs.
The S3 resource now\nsupports creating URLs using CloudFront.
The Git resource can now\ncreate tags while pushing.
Autoscrolling was broken. It’s fixed now.
Resources that are failing to check will now be shown as errored in\n the pipeline view. When viewing the resource, the check status will be\n shown on its page, and the last check error will be shown if the user\n is authenticated.
\n\nViewing an already completed build is now much less painful. Rather\nthan streaming the events in and drawing the page live, we now process\nall events and then render the build. This greatly improves the\nresponsiveness of the UI and cuts the overall rendering time by ~6x.
We have fixed a few sources of potential resource leaks in the ATC. If\nyou noticed your deployment getting slower over time before, you may\nwant to try upgrading. Symptoms include high CPU and high memory use.
Blackbox can now deliver metrics from the ATC to Datadog. To configure\nthis, colocate the Blackbox job on your web VMs (be sure to list it as\nthe first job template), and set the following property:
blackbox: \n expvar: \n datadog: \n api_key: blahblahblah\nCurrently emitted metrics are mainly focused on identifying resource\n leaks. If you’re noticing your deployment get slower over time, having\n metrics would greatly help in reporting the bug.
\n","description_markdown":"* Debugging and detecting misconfigured or failing resource checks has\n been improved.\n\n Resources that are failing to check will now be shown as errored in\n the pipeline view. When viewing the resource, the check status will be\n shown on its page, and the last check error will be shown if the user\n is authenticated.\n\n* Viewing an already completed build is now much less painful. Rather\n than streaming the events in and drawing the page live, we now process\n all events and _then_ render the build. This greatly improves the\n responsiveness of the UI and cuts the overall rendering time by ~6x.\n\n* We have fixed a few sources of potential resource leaks in the ATC. If\n you noticed your deployment getting slower over time before, you may\n want to try upgrading. Symptoms include high CPU and high memory use.\n\n* Blackbox can now deliver metrics from the ATC to Datadog. To configure\n this, colocate the Blackbox job on your web VMs (be sure to list it as\n the first job template), and set the following property:\n\n ```racket\nblackbox: \n expvar: \n datadog: \n api_key: blahblahblah\n```\n\n Currently emitted metrics are mainly focused on identifying resource\n leaks. If you’re noticing your deployment get slower over time, having\n metrics would greatly help in reporting the bug.\n","providers":[{"name":"aws","checksum":null,"checksum_type":null,"url":"https://vagrantcloud.com/concourse/boxes/lite/versions/0.42.0/providers/aws/unknown/vagrant.box"},{"name":"virtualbox","checksum":null,"checksum_type":null,"url":"https://vagrantcloud.com/concourse/boxes/lite/versions/0.42.0/providers/virtualbox/unknown/vagrant.box"}]},{"version":"0.41.0","status":"active","description_html":"\n\n\nRun
\nflysyncto upgrade Fly after deploying v0.41.0!
fly hijack -c resource-name. This should help\nwith debugging failing checks; once in the container you can directly\nrun the check by running something like this:echo '{"source":{...}}' | /opt/resource/check
Where the source reflects the configuration for the resource in your\n pipeline.
In future releases there will be better ways to detect and debug\n failing checks; this is just a first step.
\n\nHEAD when the ref\nit’s checking from becomes invalid, e.g. when a git push -f\nhappens.This will also fix any stuck resources in deployments affected by the\n bug introduced by v0.39.0 (later fixed in v0.40.0).
\n","description_markdown":"> Run `fly` `sync` to upgrade Fly after deploying v0.41.0!\n\n* The containers used for checking for new versions of resources can now\n be hijacked with `fly` `hijack` `-c` `resource-name`. This should help\n with debugging failing checks; once in the container you can directly\n run the check by running something like this:\n\n `echo '{\"source\":{...}}' | /opt/resource/check`\n\n Where the `source` reflects the configuration for the resource in your\n pipeline.\n\n In future releases there will be better ways to detect and debug\n failing checks; this is just a first step.\n\n* The Git resource will now start checking over from `HEAD` when the ref\n it’s checking from becomes invalid, e.g. when a `git` `push` `-f`\n happens.\n\n This will also fix any stuck resources in deployments affected by the\n bug introduced by v0.39.0 (later fixed in v0.40.0).\n","providers":[{"name":"aws","checksum":null,"checksum_type":null,"url":"https://vagrantcloud.com/concourse/boxes/lite/versions/0.41.0/providers/aws/unknown/vagrant.box"},{"name":"virtualbox","checksum":null,"checksum_type":null,"url":"https://vagrantcloud.com/concourse/boxes/lite/versions/0.41.0/providers/virtualbox/unknown/vagrant.box"}]},{"version":"0.40.0","status":"active","description_html":"This release fixes up a regression that affected Git repositories with\nplenty of files. Previously the resource would detect bogus versions and\nclog up the pipes. This has been fixed by upgrading Git and greatly\nsimplifying the checking implementation.
\n","description_markdown":"This release fixes up a regression that affected Git repositories with\nplenty of files. Previously the resource would detect bogus versions and\nclog up the pipes. This has been fixed by upgrading Git and greatly\nsimplifying the checking implementation.\n","providers":[{"name":"aws","checksum":null,"checksum_type":null,"url":"https://vagrantcloud.com/concourse/boxes/lite/versions/0.40.0/providers/aws/unknown/vagrant.box"},{"name":"virtualbox","checksum":null,"checksum_type":null,"url":"https://vagrantcloud.com/concourse/boxes/lite/versions/0.40.0/providers/virtualbox/unknown/vagrant.box"}]},{"version":"0.39.0","status":"active","description_html":"\n\n\nRun
\nflysyncto upgrade Fly after deploying v0.39.0!
This release adds a new way of configuring jobs called Build Plans.
\n\nWe’ll be removing support for the old style of job configuration in the\nfuture; to automatically migrate your configuration, just run\nfly configure against your instance to update your local\nconfiguration after upgrading to this release.
For more details, read on.
\n\nSince there is now more than one stage to hijack into we’ve added\n new flags for the step name (-n) and step type (-t). You can use\n these to gain shell access to any step of your build.
For more information on how you can start, see Build Plans. We’ve\n found a 43.73% increase in happiness from people who use this feature.
\n\nAs part of rolling out build plans, we now automatically translate the\n old configuration format to the new plan-based configuration.
\n\nWe’ve renamed what was formerly known as "builds" (i.e. build.yml)\nto Tasks to disambiguate from builds of a job. Jobs have builds, and\nbuilds are the result of running tasks and resource actions.
In related news, we needed to upgrade the UI to support all of these\nwonderful new flows so we’ve spruced up the build log page a little.\nThere are now individual success/failure markers for each stage and\nstages that are not-interesting (successful) will automatically\ncollapse. There are also little icons. Totally rad.
A few of you noticed that having multiple ignore paths in the\ngit-resource wasn’t really working properly. Well, we’ve fixed that.\nWe now process the ignore_paths parameter using .gitignore\nsemantics.
\n\n\nRun
\nflysyncto upgrade Fly after deploying v0.38.0!
Extra keys are now detected during config updates and rejected if\npresent. This should catch common user errors (e.g. forgetting to nest\nresource config under source) and backwards-incompatible changes\nmore safely (e.g. renaming trigger to something else).
You can now use the GitHub release resource without an access token\nfor everything except publishing a resource.
The GitHub Release resource now actually supports being used as an\ninput. We had previously forgotten to actually wire in the command.\nWoops.
\n\n\nRun
\nflysyncto upgrade Fly after deploying v0.37.0!
fly configure -c now presents the user with the changes that the\nnew configuration applies, and asks for the user to confirm.\nCommitting the configurion is done atomically, meaning confirmation\nalways applies what you expect. It will reject it if what you’ve\ncompared against has since changed, i.e. another person has run\nfly configure.
Concourse is now durable to the worker’s Garden servers going down (or\nnot being up) in the midst of builds running. This fixes EOF and\nconnection errors causing builds to error.
fly configure -c is now more helpful when it fails (it actually,\nyou know, prints the problem out, rather than silently failing).
The S3 resource now includes url metadata when fetched as an input.
Fixed an issue in the gate component’s BOSH templates, which only\nseemed to affect building the Packer box.
Added a GitHub Releases resource. It can pull down and push up\nmultiple blobs, and tracks releases via their version. For more\ndetails, see its\nREADME.
The navigation bar is now present on every page.
Upgraded to Consul v0.5. This should fix cases where the workers would\nlose contact with the ATC and never rejoin.
Fixed a panic in the ATC that would happen every time a job with no\nbuild configuration finished.
Added missing merge strategy binaries to the Git resource; this allows\nthe rebase option to work in more cases.
Now works with the latest Garden Linux again.
Reworked the build view’s ansi parsing, which now uses an external\nlibrary. This fixes some rendering issues (e.g. long swaths of text in\na single color now preserve the color for the whole region).
The faux ’back’ button on the build view page has been replaced with\nthe navigation bar from the main page.
The ATC’s logging level can be reconfigured via the API. Getting and\nsetting it is done via GET and PUT to /api/v1/log-level. Valid\nvalues are debug, info, error, and fatal.
Additional logging has been added at the debug level, now that it\n can be kept quiet by default.
Fixed a bug that could cause pending builds of serial jobs to never\nrun. Upgrading prevents this and will fix any stuck jobs.
The main page is now durable to network errors, and will continue to\nupdate.
Pending builds can now be aborted again.
Fixed a bug that caused containers not to be released after builds\ncompleted. This caused defunct containers to build up on the box and\ngive Garden a run for its money.
We’re tracking Garden master again. It’s a good day.
The Docker image resource would fail to download images over 1G in\nsize. Since some people decide to put way too much junk in their\nimages we’re increasing this to 10G. Don’t make us do it again.
Busybox, people. Busybox.
\n\nA breaking change was made to the Cloud Foundry resource. The\ncurrent_app_name field has been moved from the resource\nconfiguration to the output configuration.
The Cloud Foundry CLI used by the Cloud Foundry resource has been\nupgraded to v6.9.0.
An even more obvious effect has been chosen for the running build\nindicator. It has the advantage of testing your browser’s SVG support\nat the same time.
A build can now run even if it doesn’t have any inputs.
Input containers no longer expire if the build container takes too\nlong to initialize. This is normally due to large Docker images.
Individual resource versions can now be disabled by clicking the\ntoggle switch on the resource pages. This is useful if there is a\nbroken or pulled input resource version that you want to ignore in\nautomatically triggered builds.
The subtle pulsing animation that represented a running build has been\nreplaced by a more obvious effect.
If hijack-ing running builds isn’t your thing you can now run\nfly intercept to achieve the same thing.
The S3 resource output metadata now includes the URL of the published\nfile. This only applies to public buckets.
Removes the Turbine component. See v0.27.0’s release notes.
\n\nNote that the deployment manifest has changed once again, this time to\nremove the turbine job. See the example\nmanifests.
If you skip v0.27.0, any builds running during the Concourse upgrade\nwill be orphaned. If you’re upgrading from v0.27.0 everything should be\nfine.
\n","description_markdown":"Removes the Turbine component. See v0.27.0’s release notes.\n\nNote that the deployment manifest has changed once again, this time to\nremove the `turbine` job. See the [example\nmanifests](https://github.com/concourse/concourse/tree/2959b6cee2178421fd14cf0150798dbe66ccc9b1/manifests).\n\nIf you skip v0.27.0, any builds running _during_ the Concourse upgrade\nwill be orphaned. If you’re upgrading from v0.27.0 everything should be\nfine.\n","providers":[{"name":"virtualbox","checksum":null,"checksum_type":null,"url":"https://vagrantcloud.com/concourse/boxes/lite/versions/0.28.0/providers/virtualbox/unknown/vagrant.box"}]},{"version":"0.27.0","status":"active","description_html":"Large internal restructuring and refactoring geared towards removing the\nTurbine component.
\n\nA few things in the deployment manifest have changed as a result. See\nthe example\nmanifests\nto see what’s new.
\n\nThis new engine is better at recovering builds that were running; the\n orchestrating ATC can go down while fetching inputs, executing the\n build, or performing outputs, and recover gracefully on start. Caveat:\n if any of these steps *finish* executing while ATC is down, the\n result will be lost. This will be improved in later releases.
\n\nIn addition to being a large internal refactor, in the future this new\n engine will also support arbitrary build matrixes, and executing\n builds spanning multiple Garden backends (e.g. Linux and Windows).
\n\nA job can now consist solely of inputs and outputs, with no build\nconfigured. This is very useful for simpler jobs that just transform\nor generate artifacts.
Fixed fly hijack clobbering the terminal’s TTY state.
Garden Workers can be dynamically registered and listed through the\nAPI. Containers are distributed among them randomly.
Later this will be expanded to support workers of various platforms,\n and various supported resource types.
\n","description_markdown":"Large internal restructuring and refactoring geared towards removing the\nTurbine component.\n\nA few things in the deployment manifest have changed as a result. See\nthe [example\nmanifests](https://github.com/concourse/concourse/tree/102398a7e4701dec0fe8cdfaf99c415b64542e12/manifests)\nto see what’s new.\n\n* A new execution engine has been introduced to the ATC, which replaces\n the Turbine for running builds and checking for resources. This\n version still includes Turbine for backwards-compatibility; the next\n release will remove it.\n\n This new engine is better at recovering builds that were running; the\n orchestrating ATC can go down while fetching inputs, executing the\n build, or performing outputs, and recover gracefully on start. Caveat:\n if any of these steps \\*finish\\* executing while ATC is down, the\n result will be lost. This will be improved in later releases.\n\n In addition to being a large internal refactor, in the future this new\n engine will also support arbitrary build matrixes, and executing\n builds spanning multiple Garden backends (e.g. Linux and Windows).\n\n* A job can now consist solely of inputs and outputs, with no build\n configured. This is very useful for simpler jobs that just transform\n or generate artifacts.\n\n* Fixed `fly` `hijack` clobbering the terminal’s TTY state.\n\n* Garden Workers can be dynamically registered and listed through the\n API. Containers are distributed among them randomly.\n\n Later this will be expanded to support workers of various platforms,\n and various supported resource types.\n","providers":[{"name":"virtualbox","checksum":null,"checksum_type":null,"url":"https://vagrantcloud.com/concourse/boxes/lite/versions/0.27.0/providers/virtualbox/unknown/vagrant.box"}]},{"version":"0.26.0","status":"active","description_html":"The events protocol has been redone to better support reconnecting.\nEach event is annotated with its own version, and they are versioned\nindependently. This fixes fly hanging after the connection times\nout.
If there are no groups configured, the UI will now show everything,\nrather than nothing.
Lots of internal reworking is under way to make removing the Turbine\ncomponent easier. Events now propagate solely through the database,\nrather than proxying to the Turbine. This is done via PostgreSQL’s\nLISTEN/NOTIFY feature. Aborting builds is now also propagated via\ndatabase notifications.
This will enable future build execution engines to be implemented\n without being concerned with clustered ATC implementations.
\n\nThis new UI more truthfully represents the propagation of resources\n through the pipeline. Rather than singular labeled arrows between\n jobs, jobs now have entrance and exit lines, one per resource, and the\n resources "thread through" the UI. Hovering over a resource or a line\n will highlight all occurrences of that resource throughout the\n pipeline.
\n\nIt’s much easier to quickly collect information with the new UI. For\n example, the taller a job’s box is, the more resources it affects. You\n can also typically look one column to the left of a job to know\n exactly where its inputs are coming from.
\n\nFly now has a watch subcommand for streaming a build’s output.
The API now respects the same viewability rules as the web UI with\nregard to authentication. This makes it easier/safer to point\nthird-party build monitoring tools at your CI.
Jobs with a running build are now shown as the same color as their\nprevious state (i.e. green or red). This makes it easier to tell if\nthe pipeline is blocked - rather than looking at a monitor and seeing\nyellow, you’ll now see if it’s trying to fix a broken build, and know\nnot to push.
A Cloud Foundry Resource has been added, supporting cf push with\nblue-green deploys.
More improvements to pipeline rendering, mainly affecting\nconfigurations with groups and fan-in nodes.
Inputs to jobs can be hidden from the pipeline with hidden: true.\nThis is useful for common inputs (e.g. repositories containing\ncredentials) that feed into many jobs but aren’t really interesting\nenough to actually show. These tend to clutter up the flow, and can\nnow be hidden.
Components no longer drain indefinitely. This was a discipline kept\nthroughout development to catch deadlocked code paths, but not a great\npractice for production deployments.
They will now be given time to safely shut down, and then kill -9’d.\n All relevant code paths were audited and deemed to be interruptible,\n or fixed.
\n\nAs a result of this deploys should never hang again.
\n","description_markdown":"* More improvements to pipeline rendering, mainly affecting\n configurations with groups and fan-in nodes.\n\n* Inputs to jobs can be hidden from the pipeline with `hidden:` `true`.\n This is useful for common inputs (e.g. repositories containing\n credentials) that feed into many jobs but aren’t really interesting\n enough to actually show. These tend to clutter up the flow, and can\n now be hidden.\n\n* Components no longer drain indefinitely. This was a discipline kept\n throughout development to catch deadlocked code paths, but not a great\n practice for production deployments.\n\n They will now be given time to safely shut down, and then kill -9’d.\n All relevant code paths were audited and deemed to be interruptible,\n or fixed.\n\n As a result of this deploys should never hang again.\n","providers":[{"name":"virtualbox","checksum":null,"checksum_type":null,"url":"https://vagrantcloud.com/concourse/boxes/lite/versions/0.24.0/providers/virtualbox/unknown/vagrant.box"}]},{"version":"0.23.0","status":"active","description_html":"The groups in the pipeline view can be individually toggled with\nshift+click.
Fixed resource links from the pipeline; this was a hole in the API,\nwherein resource objects did not contain their URL. The API has been\nfixed.
Initial version supporting configuration via the Fly CLI.
\n","description_markdown":"Initial version supporting configuration via the Fly CLI.","providers":[{"name":"virtualbox","checksum":null,"checksum_type":null,"url":"https://vagrantcloud.com/concourse/boxes/lite/versions/0.20.0/providers/virtualbox/unknown/vagrant.box"}]}]}