Apache Mesos Bloghttp://mesos.apache.org/blog2020-10-20T00:00:00+00:00http://mesos.apache.org/blog/mesos-1-10-0-released/
Apache Mesos 1.10: Container Resource Bursting, Executor Domain Sockets, V1 Operator API Performance, and Reservation Update
2020-10-20T00:00:00+00:00Benjamin Mahler
<p>We are excited to announce that Apache Mesos 1.10.0 has been released! Along with lots of bug fixes and improvements, the following are the larger items:</p>
<h1>New Features and Improvements</h1>
<h2>Container Resource Bursting</h2>
<p>Mesos now supports per-container CPU and memory usage beyond the allocation to the container, up to a configured limit. Previously, it was only possible to enable CPU bursting for all (or none) of the containers on the agent, without per-container control.</p>
<p>This feature is useful for batch style workloads that can benefit from getting access to any unused CPU or memory on the agent. It’s also helpful for increasing utilization of the machines in a cluster, by allowing unallocated and/or unused CPU to be utilized.</p>
<p>Frameworks specify CPU and memory limits for tasks (separately from resource requests) and also the level of isolation they desire when launching task groups - CPU and memory may be isolated at the executor container level, or the task container level.</p>
<p>See <a href="http://mesos.apache.org/documentation/latest/running-workloads/">here</a> for more information.</p>
<h2>Executor Domain Sockets</h2>
<p>When a task is launched in Mesos, the executor is expected to connect to the agent through a TCP socket in order to register itself. This makes it necessary to allow TCP connections between the environment of the executor and the host network of the mesos agent. In some contexts (e.g. containers using CNI networks), it is desirable to firewall any TCP connections to the agent host. To support this use-case, Mesos 1.10.0 introduces the ability for executors to communicate with agents on the same host using unix domain sockets.</p>
<p>See <a href="http://mesos.apache.org/documentation/latest/executor-http-api/">here</a> for more information on how to use this feature.</p>
<h2>V1 Operator API Performance</h2>
<p>V1 operator API performance has lagged behind the V0 operator API, as the V0 operator API underwent significant performance optimizations in recent releases. This has made it impossible to recommend the V1 operator API to users given its scalability issues.</p>
<p>Performance of read-only V1 operator API calls has been improved by introducing direct serialization into JSON/protobuf and extending the batching mechanism to parallel processing of these calls by the master (similarly to <code>/state</code> endpoint). The V1 operator API performance is now on par with the V0 HTTP endpoints.</p>
<h2>Reservation Update</h2>
<p>Reservations in Mesos control which frameworks can consume resources, e.g., operators use reservations to partition clusters, or frameworks use DYNAMIC reservations to ensure that resources are not offered elsewhere should a task fail over. In addition, persistent volumes are bound to a reservation.</p>
<p>Existing reservations can now be updated via the RESERVE_RESOURCES master API call. This, for example, allows the operator to move persistent volumes between roles non-destructively.</p>
<p>An example is shown <a href="http://mesos.apache.org/documentation/latest/operator-http-api/#reserve_resources">here</a>.</p>
<h1>Upgrade</h1>
<p>Upgrades from Mesos 1.9.0 to Mesos 1.10.0 should be straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.10.0.</p>
<h1>Community</h1>
<p>Inspired by the work that went into this release? Want to get involved? Have feedback? We’d love to hear from you! Join a <a href="http://mesos.apache.org/community/#working-groups">working group</a> or start a conversation in the <a href="http://mesos.apache.org/community/">community</a>!</p>
<h1>Thank you!</h1>
<p>Special thanks to Andrei Sekretenko as the release manager.</p>
<p>Thanks to the 24 contributors who made Mesos 1.10.0 possible:</p>
<p>Adam Cecile, Aleksandr Kuzmitsky, Andrei Budnik, Andrei Sekretenko, Benjamin Bannier, Benjamin Mahler, Benno Evers, Bo Anderson, Charles-Francois Natali, Chun-Hung Hsiao, Damien Gerard, Dominik Dary, Dong Zhu, Greg Mann, Grégoire Seux, James Peach, James Wright, Jonathan Robson, Joseph Wu, Maxime Brugidou, Meng Zhu, Qian Zhang, Vinod Kone</p>
http://mesos.apache.org/blog/mesos-1-9-0-released/
Apache Mesos 1.9: Agent Draining, Quota Limit and Security Improvements
2019-09-05T00:00:00+00:00Qian Zhang & Gilbert Song
<p>We are excited to announce that Apache Mesos 1.9.0 is now available for <a href="/downloads">download</a>. Please take a look at what’s new in this release!</p>
<h1>New Features and Improvements</h1>
<h2>Agent Draining</h2>
<p>Automatic agent draining was added to allow operators to prepare agent nodes for maintenance without requiring schedulers to implement support for the feature. Since the pre-existing maintenance primitives offered by Mesos require that schedulers make changes, some operators have had difficulty using them effectively in clusters containing frameworks which have not done so. When automatic draining is initiated on an agent, all tasks are gracefully killed, and operators can monitor the master’s state to detect when draining on the agent is complete.</p>
<p>Agent deactivation and reactivation primitives were also added to the master API, allowing operators to stop and resume offers from particular agents. Used in concert with framework-specific APIs, this new functionality enables operators to perform manual draining of agent nodes in cases where greater control is desired.</p>
<h2>Resource Management</h2>
<p>Prior to Mesos 1.9, the quota related APIs only exposed quota “guarantees” which ensured a minimum amount of resources would be available to a role. Setting guarantees also set implicit quota limits. In Mesos 1.9.0, quota limits are now exposed directly.</p>
<ul>
<li><p>Quota guarantees are now deprecated in favor of using only quota limits. Enforcement of quota guarantees required that Mesos holds back enough resources to meet all of the unsatisfied quota guarantees. Since Mesos is moving towards an optimistic offer model (to improve multi-role / multi- scheduler scalability, see MESOS-1607), it will become no longer possible to enforce quota guarantees by holding back resources. In such a model, quota limits are simple to enforce, but quota guarantees would require a complex “effective limit” propagation model to leave space for unsatisfied guarantees.</p></li>
<li><p>For these reasons, quota guarantees, while still functional in Mesos 1.9, are now deprecated. A combination of limits and priority based preemption will be simpler in an optimistic offer model.</p></li>
</ul>
<h2>Containerization</h2>
<p>A number of containerization-related improvements have landed in Mesos 1.9.0:</p>
<ul>
<li><p>The Mesos containerizer now supports configurable IPC namespace and /dev/shm. Container can be configured to have a private IPC namespace and /dev/shm or share them from its parent, and the size of its private /dev/shm is also configurable.</p></li>
<li><p>A new <code>/containerizer/debug</code> HTTP endpoint has been added. This endpoint exposes debug information for the Mesos containerizer. At the moment, it returns a list of pending operations related to isolators and launchers.</p></li>
<li><p>A new Linux NNP (No New Privs) isolator has been added to the Mesos Containerizer. The isolator allows configuration of the <a href="https://www.kernel.org/doc/Documentation/prctl/no_new_privs.txt">no_new_privs</a> flag for launched containers. The <code>no_new_privs</code> flag disables the ability of container tasks to acquire additional privileges by means of executing a child process e.g. through invocation of <code>setuid</code> or <code>setgid</code> programs. The flag is configurable on the agent and provides additional depth of security for containerized processes.</p></li>
<li><p>A new <code>--docker_ignore_runtime</code> flag has been added. This causes the agent to ignore any runtime configuration present in Docker images.</p></li>
<li><p>The Mesos containerizer now includes ephemeral overlayfs storage in the task disk quota as well as sandbox storage.</p></li>
</ul>
<h2>Improved Security for TLS Connections</h2>
<p>Since Mesos 0.23, Mesos had support for using TLS [1] to encrypt the communication to and from Mesos components - the same protocol that secures <code>https</code>, <code>smtps</code>, and many others. Roughly speaking, every time a TLS client connects to a TLS server, that server will present a certificate signed by a trusted certificate authority which is used to verify the identity of the server.</p>
<p>In Mesos, this behaviour is controlled by the environment variables <code>LIBPROCESS_SSL_VERIFY_CERT</code> and <code>LIBPROCESS_SSL_REQUIRE_CERT</code>. The former would do the cryptographic verification <strong>if</strong> a certificate was supplied, and the latter would reject all connections where no certificate was presented. This may sound straightforward, but this behaviour has proven challenging for Mesos operators, with many leaving TLS verification disabled in practice. The reason for that is that Mesos components are acting as both TLS client and server at the same time.</p>
<p>Enabling server certificate validation in this scenario had the effect of requiring <strong>all</strong> incoming connections to present valid client certificates. This put an additional burden on operators to build infrastructure to distribute valid client certificates to all users of Mesos endpoints.</p>
<p>With Mesos 1.9, we updated the semantics of both flags to be more aligned with the needs of Mesos operators:</p>
<ul>
<li><p><code>LIBPROCESS_SSL_VERIFY_CERT</code> now only applies to <em>server certificates</em>, which are always required for TLS connections. If it is set to true, the server certificate is verified for all outgoing connections.</p></li>
<li><p><code>LIBPROCESS_SSL_REQUIRE_CERT</code> now only applies to <em>client certificates</em>: If it is set to true, all incoming connections must present a valid client certificate.</p></li>
</ul>
<p>By switching to the OpenSSL-provided API for hostname validation [2], we are able to improve security and make the behaviour more uniform across different platforms. We were also able to eliminate reverse DNS lookups while establishing a connection which improves reliability and performance.</p>
<p>[1] https://en.wikipedia.org/wiki/Transport_Layer_Security</p>
<p>[2] http://mesos.apache.org/documentation/latest/ssl/#libprocess_ssl_hostname_validation_scheme-legacy-openssl-default</p>
<h1>Upgrade</h1>
<p>Upgrades from Mesos 1.8.0 to Mesos 1.9.0 should be straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.9.0.</p>
<h1>Community</h1>
<p>Inspired by the work that went into this release? Want to get involved? Have feedback? We’d love to hear from you! Join a <a href="http://mesos.apache.org/community/#working-groups">working group</a> or start a conversation in the <a href="http://mesos.apache.org/community/">community</a>!</p>
<h1>Thank you!</h1>
<p>Thanks to the 28 contributors who made Mesos 1.9.0 possible:</p>
<p>Alexander Rukletsov, Andrei Budnik, Andrei Sekretenko, Armand Grillet, Bartosz Galek, Benjamin Bannier, Benjamin Mahler, Benno Evers, Bilal Amarni, Chun-Hung Hsiao, Gastón Kleiman, Gilbert Song, Greg Mann, Hans Beck, Jacob Janco, James Peach, James Wright, Jan Schlicht, Joseph Wu, Meng Zhu, Pavel Kirillov, Qian Zhang, Stéphane Cottin, Till Toenshoff, Tomasz Janiszewski, Vinod Kone, Zhitao Li, Fei Long</p>
http://mesos.apache.org/blog/mesos-1-8-1-released/
Apache Mesos 1.8.1 Released
2019-07-29T00:00:00+00:00Benno Evers
<p>The latest Mesos 1.8.x release, 1.8.1, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes important bug fixes and improvements on top of 1.8.0. It is recommended to use this version if you are considering using Mesos 1.8. More specifically, this release includes the following:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9395">MESOS-9395</a> - Check failure on <code>StorageLocalResourceProviderProcess::applyCreateDisk</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9616">MESOS-9616</a> - <code>Filters.refuse_seconds</code> declines resources not in offers.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9730">MESOS-9730</a> - Executors cannot reconnect with agents using TLS1.3</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9750">MESOS-9750</a> - Agent V1 <code>GET_STATE</code> response may report a complete executor’s tasks as non-terminal after a graceful agent shutdown.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9766">MESOS-9766</a> - /<strong>processes</strong> endpoint can hang.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9779">MESOS-9779</a> - <code>UPDATE_RESOURCE_PROVIDER_CONFIG</code> agent call returns 404 ambiguously.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9782">MESOS-9782</a> - Random sorter fails to clear removed clients.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9786">MESOS-9786</a> - Race between two <code>REMOVE_QUOTA</code> calls crashes the master.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9803">MESOS-9803</a> - Memory leak caused by an infinite chain of futures in <code>UriDiskProfileAdaptor</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9831">MESOS-9831</a> - Master should not report disconnected resource providers.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9852">MESOS-9852</a> - Slow memory growth in master due to deferred deletion of offer filters and timers.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9856">MESOS-9856</a> - REVIVE call with specified role(s) clears filters for all roles of a framework.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9870">MESOS-9870</a> - Simultaneous adding/removal of a role from framework’s roles and its suppressed roles crashes the master.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9695">MESOS-9695</a> - Remove the duplicate pid check in Docker containerizer</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9759">MESOS-9759</a> - Log required quota headroom and available quota headroom in the allocator.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9787">MESOS-9787</a> - Log slow SSL (TLS) peer reverse DNS lookup.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.8.1">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.8.0 cluster to Mesos 1.8.1 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.8.1 from 1.7.x, 1.6.x, or 1.5.x.</p>
<h3>Try it out</h3>
<p>Please try out this release and let us know what you think. If you run into any issues, let us know on the <a href="https://mesos.apache.org/community">user mailing list and/or Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 9 contributors who made 1.8.1 possible:</p>
<p>Andrei Budnik, Andrei Sekretenko, Benjamin Mahler, Chun-Hung Hsiao, Gilbert Song, Joseph Wu, Meng Zhu, Qian Zhang, Stéphane Cottin</p>
http://mesos.apache.org/blog/mesos-1-8-0-released/
Apache Mesos 1.8: Allocator Performance, Seccomp Isolation and Operation Feedback
2019-05-02T00:00:00+00:00Benno Evers
<p>We’re pleased to announce that Mesos 1.8.0 is now available for <a href="/downloads">download</a>.</p>
<p>As usual, lots of work and care went into this release, with a total of 255 JIRA issues resolved,
and 1149 commits containing a total diffstat of 656 files changed, 62213 insertions(+) and 20403 deletions(-).</p>
<pre><code> 4.9% 3rdparty/
1.5% 3rdparty/libprocess/
1.6% 3rdparty/stout/
2.3% docs/
2.5% include/
2.0% site/
77.4% src/
7.8% src/csi/
3.6% src/examples/
11.0% src/master/
4.2% src/resource_provider/
10.3% src/slave/
28.8% src/tests/
6.7% support/
3.7% support/python3/
</code></pre>
<p>Looking at the distribution of changes, nearly all components have seen major activity.
Some of that is presented in more detail below.</p>
<p>One major focus for the 1.8 contributors seems to have been solid test coverage, with
nearly one third of the total amount of lines changed being inside the test suite.</p>
<h2>Highlights</h2>
<h3>Allocator Performance Improvements</h3>
<p>In Mesos 1.8, allocator cycle time is significantly decreased when quota is
used; around 40% for a small size cluster and up to 70% for larger clusters.
This greatly narrows the allocator performance gap between quota and non-quota
usage scenarios.</p>
<p>In addition, schedulers can now specify the minimum resource quantities needed
in an offer, which acts as an override of the global <code>--min_allocatable_resources</code>
master flag. Updating schedulers to specify this field improves multi-scheduler
scalability as it reduces the amount of offers declined from having insufficient
resource quantities.</p>
<p>Note that this feature currently requires that the scheduler re-subscribes each
time it wants to mutate the minimum resource quantity offer filter information,
see <a href="https://issues.apache.org/jira/browse/MESOS-7258">MESOS-7258</a>.</p>
<h3>Seccomp Isolator</h3>
<p>A new <code>linux/seccomp</code> <a href="docs/isolators/linux-seccomp">isolator</a> was added. This
isolator makes use of the
<a href="https://www.kernel.org/doc/Documentation/prctl/seccomp_filter.txt">seccomp</a>
facility provided by recent linux kernels.</p>
<p>Using this isolator, containers launched by Mesos containerizer can be sandboxed
by enabling filtering of system calls using a configurable policy.</p>
<h3>Operation Feedback</h3>
<p>In Mesos 1.6, operation feedback was introduced for operations on resources
by a resource provider.</p>
<p>In Mesos 1.8, v1 schedulers can now receive operation feedback for operations
on agent default resources, i.e. normal cpu, memory, and disk. This means that
the v1 scheduler API’s operation feedback feature can now be used for any offer
operations except for <code>LAUNCH</code> and <code>LAUNCH_GROUP</code>, on any type of resources.</p>
<h3>Containerization</h3>
<p>A number of containerization-related improvements have landed in Mesos 1.8:</p>
<ul>
<li><p>Support pulling docker images with docker manifest
V2 Schema2 on Mesos Containerizer.</p></li>
<li><p>Support custom port range option to the <code>network/ports</code>
isolator. Added the <code>--container_ports_isolated_range</code> flag to the
<code>network/ports</code> isolator. This allows the operator to specify a custom
port range to be protected by the isolator.</p></li>
<li><p>Support XFS quota for persistent volumes. Added
persistent volume support to the <code>disk/xfs</code> isolator.</p></li>
<li><p>Support an option to create non-existing host
paths for host path volume in Mesos Containerizer. Added a new
agent flag <code>--host_path_volume_force_creation</code> for the
<code>volume/host_path</code> isolator.</p></li>
</ul>
<h3>CLI Improvements</h3>
<p>The Mesos CLI now offers the task subcommand with two actions. The first
action, <code>task attach</code>, allows you to attach your terminal to a running
task launched with a tty. The second action, <code>task exec</code>, launches a
new nested container inside a running task.</p>
<p>To build the CLI, use the flag <code>--enable-new-cli</code> with Autotools or
<code>-DENABLE_NEW_CLI=1</code> with CMake on MacOS or Linux.</p>
<h3>Experimental Features</h3>
<p>Mesos 1.8 adds experimental support for the new CSI v1 API. Operators can deploy
plugins that are compatible to either CSI v0 or v1 to create persistent
volumes through storage local resource providers, and Mesos will
automatically detect which CSI versions are supported by the plugins.</p>
<h2>Upgrade</h2>
<p>Upgrades from Mesos 1.7.0 to Mesos 1.8.0 should be straightforward. Please
refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a>
for detailed information on upgrading to Mesos 1.8.0.</p>
<ul>
<li>Frameworks relying on the experimental <code>RECONCILE_OPERATIONS</code> call can
not be updated to Mesos 1.8, since the API of that has been reworked in
a non-backwards compatible manner.</li>
</ul>
<h2>Credits</h2>
<p>Finally, a big <strong>THANK YOU</strong> goes out to all the 49 patch authors who made the 1.8.0 release possible:</p>
<p>Aaron Wood, Alexander Rojas, Alexander Rukletsov, Andrei Budnik, Andrei Sekretenko, Andrew Schwartzmeyer,
Armand Grillet, Benjamin Bannier, Benjamin Hindman, Benjamin Mahler, Benno Evers, Chun-Hung Hsiao,
Clement Michaud, Deepak Goel, Dominik Dary, Dragos Schebesch, Eric Chung, Fei Long, Gastón Kleiman,
Gilbert Song, Greg Mann, Ilya Pronin, Jacob Janco, James DeFelice, James Peach, Jan Schlicht,
Jason Lai, Jie Yu, Joseph Wu, Kapil Arya, Kevin Klues, Liangyu Zhao, Meng Zhu, Michael Park, Michał Łowicki,
Packt, Pavel Kirillov, Qian Zhang, Robin Gögge, Se Choi, Senthil Kumaran, Sergey Urbanovich, Tad Guo
Till Toenshoff, Tomasz Janiszewski, Vinod Kone, Xudong Ni</p>
http://mesos.apache.org/blog/mesos-1-5-3-released/
Apache Mesos 1.5.3 Released
2019-03-19T00:00:00+00:00Gilbert Song
<p>The latest Mesos 1.5.x release, 1.5.3, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes important bug fixes and improvements on top of 1.5.0. It is recommended to use this version if you are considering using Mesos 1.5. More specifically, this release includes the following:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7474">MESOS-7474</a> - Mesos fetcher cache doesn’t retry when missed.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8887">MESOS-8887</a> - Unreachable tasks are not GC'ed when unreachable agent is GC'ed.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8907">MESOS-8907</a> - Docker image fetcher fails with HTTP/2.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9210">MESOS-9210</a> - Mesos v1 scheduler library does not properly handle SUBSCRIBE retries.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9305">MESOS-9305</a> - Create cgoup recursively to workaround systemd deleting cgroups_root.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9317">MESOS-9317</a> - Some master endpoints do not handle failed authorization properly.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9332">MESOS-9332</a> - Nested container should run as the same user of its parent container by default.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9334">MESOS-9334</a> - Container stuck at ISOLATING state due to libevent poll never returns.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9362">MESOS-9362</a> - Test <code>CgroupsIsolatorTest.ROOT_CGROUPS_CreateRecursively</code> is flaky.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9411">MESOS-9411</a> - Validation of JWT tokens using HS256 hashing algorithm is not thread safe.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9492">MESOS-9492</a> - Persist CNI working directory across reboot.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9507">MESOS-9507</a> - Agent could not recover due to empty docker volume checkpointed files.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9518">MESOS-9518</a> - CNI_NETNS should not be set for orphan containers that do not have network namespace.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9532">MESOS-9532</a> - ResourceOffersTest.ResourceOfferWithMultipleSlaves is flaky.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9533">MESOS-9533</a> - CniIsolatorTest.ROOT_CleanupAfterReboot is flaky.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9555">MESOS-9555</a> - Allocator CHECK failure: reservationScalarQuantities.contains(role).</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9581">MESOS-9581</a> - Mesos package naming appears to be undeterministic.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.5.3">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.5.0 cluster to Mesos 1.5.3 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.5.3 from 1.4.x, 1.3.x, or 1.2.x.</p>
<h3>Try it out</h3>
<p>Please try out this release and let us know what you think. If you run into any issues, let us know on the <a href="https://mesos.apache.org/community">user mailing list and/or Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 7 contributors who made 1.5.3 possible:</p>
<p>Andrei Budnik, Benjamin Mahler, Gilbert Song, Jie Yu, Qian Zhang, Till Toenshoff, Vinod Kone</p>
http://mesos.apache.org/blog/mesos-1-4-3-released/
Apache Mesos 1.4.3 Released
2019-02-26T00:00:00+00:00Meng Zhu
<p>The latest Mesos 1.4.x release, 1.4.3, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes important bug fixes and improvements on top of 1.4.2. It is recommended to use this version if you are considering using Mesos 1.4. More specifically, this release includes the following notable fixes:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8128">MESOS-8128</a> - Make os::pipe file descriptors O_CLOEXEC.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8568">MESOS-8568</a> - Command checks should always call <code>WAIT_NESTED_CONTAINER</code> before <code>REMOVE_NESTED_CONTAINER</code></li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8620">MESOS-8620</a> - Containers stuck in FETCHING possibly due to unresponsive server.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8917">MESOS-8917</a> - Agent leaking file descriptors into forked processes.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8921">MESOS-8921</a> - Autotools don’t work with newer OpenJDK versions</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9144">MESOS-9144</a> - Master authentication handling leads to request amplification.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9145">MESOS-9145</a> - Master has a fragile burned-in 5s authentication timeout.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9146">MESOS-9146</a> - Agent has a fragile burn-in 5s authentication timeout.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9147">MESOS-9147</a> - Agent and scheduler driver authentication retry backoff time could overflow.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9151">MESOS-9151</a> - Container stuck at ISOLATING due to FD leak.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9170">MESOS-9170</a> - Zookeeper doesn’t compile with newer gcc due to format error.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9196">MESOS-9196</a> - Removing rootfs mounts may fail with EBUSY.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9221">MESOS-9221</a> - If some image layers are large, the image pulling may stuck due to the authorized token expired.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9231">MESOS-9231</a> - <code>docker inspect</code> may return an unexpected result to Docker executor due to a race condition.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9279">MESOS-9279</a> - Docker Containerizer ‘usage’ call might be expensive if mount table is big.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9283">MESOS-9283</a> - Docker containerizer actor can get backlogged with large number of containers.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9304">MESOS-9304</a> - Test <code>CGROUPS_ROOT_PidNamespaceForward</code> and <code>CGROUPS_ROOT_PidNamespaceBackward</code> fails on 1.4.x.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9334">MESOS-9334</a> - Container stuck at ISOLATING state due to libevent poll never returns.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9419">MESOS-9419</a> - Executor to framework message crashes master if framework has not re-registered.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9480">MESOS-9480</a> - Master may skip processing authorization results for <code>LAUNCH_GROUP</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9492">MESOS-9492</a> - Persist CNI working directory across reboot.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9501">MESOS-9501</a> - Mesos executor fails to terminate and gets stuck after agent host reboot.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9502">MESOS-9502</a> - IOswitchboard cleanup could get stuck due to FD leak from a race.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9518">MESOS-9518</a> - CNI_NETNS should not be set for orphan containers that do not have network namespace.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9532">MESOS-9532</a> - ResourceOffersTest.ResourceOfferWithMultipleSlaves is flaky.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9533">MESOS-9533</a> - CniIsolatorTest.ROOT_CleanupAfterReboot is flaky.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9510">MESOS-9510</a> - Disallowed nan, inf and so on in <code>Value::Scalar</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9516">MESOS-9516</a> - Extend <code>min_allocatable_resources</code> flag to cover non-scalar resources.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.4.3">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.4.2 cluster to Mesos 1.4.3 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.4.2 from 1.0.x, 1.2.x, or 1.3.x.</p>
<h3>Try it out</h3>
<p>Please try out this release and let us know what you think. If you run into any issues, let us know on the <a href="https://mesos.apache.org/community">user mailing list and/or Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 18 contributors who made 1.4.3 possible:</p>
<p>Alexander Rukletsov, Andrei Budnik, Andrew Schwartzmeyer, Benjamin Bannier, Benjamin Mahler, Benno Evers, Chun-Hung Hsiao, Deepak Goel, Gastón Kleiman, Gilbert Song, Greg Mann, James Peach, Jie Yu, Kapil Arya, Meng Zhu, Michael Park, Qian Zhang, Radhika Jandhyala</p>
http://mesos.apache.org/blog/mesos-1-7-1-released/
Apache Mesos 1.7.1 Released
2019-01-28T00:00:00+00:00Chun-Hung Hsiao & Gastón Kleiman
<p>Apache Mesos 1.7.1 is now available for <a href="http://mesos.apache.org/downloads">download</a>. As we continue to focus on performance, the community can get access to the latest performance improvements in this release. The <a href="http://mesos.apache.org/documentation/latest/csi/">experimental support for Container Storage Interface (CSI)</a> has been improved in this release as well. Last but not least, this release includes 38 bug fixes to 1.7.0. If you are considering using Mesos 1.7, it is recommended to use 1.7.1.</p>
<p>Specifically, the following critical bugs are resolved in this release:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9131">MESOS-9131</a> - Health checks launching nested containers while a container is being destroyed lead to unkillable tasks.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9228">MESOS-9228</a> - SLRP does not clean up plugin containers after it is removed.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9279">MESOS-9279</a> - Docker Containerizer ‘usage’ call might be expensive if mount table is big.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9281">MESOS-9281</a> - SLRP gets a stale checkpoint after system crash.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9283">MESOS-9283</a> - Docker containerizer actor can get backlogged with large number of containers.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9308">MESOS-9308</a> - URI disk profile adaptor could deadlock.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9317">MESOS-9317</a> - Some master endpoints do not handle failed authorization properly.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9334">MESOS-9334</a> - Container stuck at ISOLATING state due to libevent poll never returns.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9419">MESOS-9419</a> - Executor to framework message crashes master if framework has not re-registered.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9474">MESOS-9474</a> - Master does not respect authorization result for <code>CREATE_DISK</code> and <code>DESTROY_DISK</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9480">MESOS-9480</a> - Master may skip processing authorization results for <code>LAUNCH_GROUP</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9501">MESOS-9501</a> - Mesos executor fails to terminate and gets stuck after agent host reboot.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9502">MESOS-9502</a> - IOswitchboard cleanup could get stuck due to FD leak from a race.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9508">MESOS-9508</a> - Official 1.7.0 tarball can’t be built on Ubuntu 16.04 LTS.</li>
</ul>
<p>And this release includes the following critical improvements:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6765">MESOS-6765</a> - “Make the Resources wrapper ”“copy-on-write”“ to improve performance.”</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9239">MESOS-9239</a> - Improve sorting performance in the DRF sorter.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9249">MESOS-9249</a> - Avoid dirtying the DRF sorter when allocating resources.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9275">MESOS-9275</a> - Allow optional <code>profile</code> to be specified in <code>CREATE_DISK</code> offer operation.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9305">MESOS-9305</a> - Create cgoup recursively to workaround systemd deleting cgroups_root.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9321">MESOS-9321</a> - Add an optional <code>vendor</code> field in <code>Resource.DiskInfo.Source</code>.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.7.1">CHANGELOG</a>.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.7.0 cluster to Mesos 1.7.1 are straightforward. However, if your framework is consuming the <a href="http://mesos.apache.org/documentation/latest/csi/">experimental support for CSI</a>, the language binding used by the framework should be updated.</p>
<p>For detailed information on upgrading to Mesos 1.7.1 from other versions, please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a>.</p>
<h3>Try it out</h3>
<p>Please try out this release and let us know what you think. If you run into any issues, let us know on the <a href="https://mesos.apache.org/community">user mailing list and/or Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 20 contributors who made 1.7.1 possible:</p>
<p>Alexander Rojas, Alexander Rukletsov, Andrei Budnik, Benjamin Bannier, Benjamin Mahler, Benno Evers, Chun-Hung Hsiao, Deepak Goel, Gastón Kleiman, Gilbert Song, Greg Mann, James Peach, Jie Yu, Joseph Wu, Kevin Klues, Meng Zhu, Qian Zhang, Till Toenshoff, Vinod Kone, Fei Long</p>
http://mesos.apache.org/blog/mesos-1-5-2-released/
Apache Mesos 1.5.2 Released
2019-01-24T00:00:00+00:00Gilbert Song
<p>The latest Mesos 1.5.x release, 1.5.2, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes important bug fixes and improvements on top of 1.5.0. It is recommended to use this version if you are considering using Mesos 1.5. More specifically, this release includes the following:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-3790">MESOS-3790</a> - ZooKeeper connection should retry on <code>EAI_NONAME</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7474">MESOS-7474</a> - Mesos fetcher cache doesn’t retry when missed.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8128">MESOS-8128</a> - Make os::pipe file descriptors O_CLOEXEC.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8418">MESOS-8418</a> - mesos-agent high cpu usage because of numerous /proc/mounts reads.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8545">MESOS-8545</a> - AgentAPIStreamingTest.AttachInputToNestedContainerSession is flaky.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8568">MESOS-8568</a> - Command checks should always call <code>WAIT_NESTED_CONTAINER</code> before <code>REMOVE_NESTED_CONTAINER</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8620">MESOS-8620</a> - Containers stuck in FETCHING possibly due to unresponsive server.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8830">MESOS-8830</a> - Agent gc on old slave sandboxes could empty persistent volume data.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8871">MESOS-8871</a> - Agent may fail to recover if the agent dies before image store cache checkpointed.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8904">MESOS-8904</a> - Master crash when removing quota.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8906">MESOS-8906</a> - <code>UriDiskProfileAdaptor</code> fails to update profile selectors.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8907">MESOS-8907</a> - Docker image fetcher fails with HTTP/2.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8917">MESOS-8917</a> - Agent leaking file descriptors into forked processes.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8921">MESOS-8921</a> - Autotools don’t work with newer OpenJDK versions.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8935">MESOS-8935</a> - Quota limit “chopping” can lead to cpu-only and memory-only offers.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8936">MESOS-8936</a> - Implement a Random Sorter for offer allocations.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8942">MESOS-8942</a> - Master streaming API does not send (health) check updates for tasks.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8945">MESOS-8945</a> - Master check failure due to CHECK_SOME(providerId).</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8947">MESOS-8947</a> - Improve the container preparing logging in IOSwitchboard and volume/secret isolator.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8952">MESOS-8952</a> - process::await/collect n<sup>2</sup> performance issue.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8963">MESOS-8963</a> - Executor crash trying to print container ID.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8978">MESOS-8978</a> - Command executor calling setsid breaks the tty support.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8980">MESOS-8980</a> - mesos-slave can deadlock with docker pull.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8986">MESOS-8986</a> - <code>slave.available()</code> in the allocator is expensive and drags down allocation performance.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8987">MESOS-8987</a> - Master asks agent to shutdown upon auth errors.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9024">MESOS-9024</a> - Mesos master segfaults with stack overflow under load.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9049">MESOS-9049</a> - Agent GC could unmount a dangling persistent volume multiple times.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9116">MESOS-9116</a> - Launch nested container session fails due to incorrect detection of <code>mnt</code> namespace of command executor’s task.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9125">MESOS-9125</a> - Port mapper CNI plugin might fail with “Resource temporarily unavailable”.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9127">MESOS-9127</a> - Port mapper CNI plugin might deadlock iptables on the agent.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9131">MESOS-9131</a> - Health checks launching nested containers while a container is being destroyed lead to unkillable tasks.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9142">MESOS-9142</a> - CNI detach might fail due to missing network config file.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9144">MESOS-9144</a> - Master authentication handling leads to request amplification.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9145">MESOS-9145</a> - Master has a fragile burned-in 5s authentication timeout.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9146">MESOS-9146</a> - Agent has a fragile burn-in 5s authentication timeout.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9147">MESOS-9147</a> - Agent and scheduler driver authentication retry backoff time could overflow.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9151">MESOS-9151</a> - Container stuck at ISOLATING due to FD leak.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9170">MESOS-9170</a> - Zookeeper doesn’t compile with newer gcc due to format error.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9196">MESOS-9196</a> - Removing rootfs mounts may fail with EBUSY.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9231">MESOS-9231</a> - <code>docker inspect</code> may return an unexpected result to Docker executor due to a race condition.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9267">MESOS-9267</a> - Mesos agent crashes when CNI network is not configured but used.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9279">MESOS-9279</a> - Docker Containerizer ‘usage’ call might be expensive if mount table is big.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9283">MESOS-9283</a> - Docker containerizer actor can get backlogged with large number of containers.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9305">MESOS-9305</a> - Create cgoup recursively to workaround systemd deleting cgroups_root.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9308">MESOS-9308</a> - URI disk profile adaptor could deadlock.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9317">MESOS-9317</a> - Some master endpoints do not handle failed authorization properly.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9332">MESOS-9332</a> - Nested container should run as the same user of its parent container by default.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9334">MESOS-9334</a> - Container stuck at ISOLATING state due to libevent poll never returns.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9411">MESOS-9411</a> - Validation of JWT tokens using HS256 hashing algorithm is not thread safe.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9419">MESOS-9419</a> - Executor to framework message crashes master if framework has not re-registered.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9480">MESOS-9480</a> - Master may skip processing authorization results for <code>LAUNCH_GROUP</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9492">MESOS-9492</a> - Persist CNI working directory across reboot.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9501">MESOS-9501</a> - Mesos executor fails to terminate and gets stuck after agent host reboot.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9502">MESOS-9502</a> - IOswitchboard cleanup could get stuck due to FD leak from a race.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9510">MESOS-9510</a> - Disallowed nan, inf and so on in <code>Value::Scalar</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9516">MESOS-9516</a> - Extend <code>min_allocatable_resources</code> flag to cover non-scalar resources.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9518">MESOS-9518</a> - CNI_NETNS should not be set for orphan containers that do not have network namespace.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.5.2">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.5.0 cluster to Mesos 1.5.2 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.5.2 from 1.4.x, 1.3.x, or 1.2.x.</p>
<h3>Try it out</h3>
<p>Please try out this release and let us know what you think. If you run into any issues, let us know on the <a href="https://mesos.apache.org/community">user mailing list and/or Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 29 contributors who made 1.5.2 possible:</p>
<p>Alexander Rojas, Alexander Rukletsov, Andrei Budnik, Andrew Schwartzmeyer, Armand Grillet, Benjamin Bannier, Benjamin Mahler, Benno Evers, Chun-Hung Hsiao, Deepak Goel, Gastón Kleiman, Gilbert Song, Greg Mann, James Peach, Jie Yu, Joseph Wu, Kapil Arya, Kevin Klues, Kjetil Joergensen, Meng Zhu, Michael Park, Qian Zhang, Radhika Jandhyala, Till Toenshoff, Vinod Kone, Zhitao Li, bin zheng, fei long, he yi hua</p>
http://mesos.apache.org/blog/mesos-mini/
Run Mesos Locally with Mesos Mini Docker Container
2018-11-19T00:00:00+00:00Jie Yu
<p><a href="https://hub.docker.com/r/mesos/mesos-mini/">Mesos Mini</a> is a Docker image maintained by the Apache Mesos community.
It allows you to test Mesos locally with a simple <code>docker run</code>.</p>
<h1>Why Mesos Mini</h1>
<p>Being able to spin up a local Mesos cluster in Docker can greatly simplify the work in the following scenarios:</p>
<ul>
<li><em>Demo</em>: Imagine doing a live demo with Mesos in a conference with unstable Wifi.</li>
<li><em>Framework development</em>: Write end-to-end integration tests for your framework with a local Mesos cluster in a Docker container.
This can be easily automated in your test suite.</li>
<li><em>Test new Mesos features</em>: Test new features from Mesos that haven’t been released yet.
You might be able to do that by building Mesos from the source code, but most framework developers do not know how to do it, and it is slow.</li>
</ul>
<p>The idea is similar to <a href="https://github.com/kubernetes/minikube">minikube</a> or <a href="https://github.com/ContainerSolutions/minimesos">minimesos</a>.</p>
<p>However, <a href="https://github.com/ContainerSolutions/minimesos">minimesos</a> is no longer maintained.
As a result, Apache Mesos community decides to maintain a solution in Mesos repository to simplify CI integrations.</p>
<h1>Get started</h1>
<p>Make sure <a href="https://docs.docker.com/install/">Docker</a> is installed.
We have tested on both Linux and MacOS.</p>
<p>To create a local Mesos cluster, simply do a <code>docker run</code>:</p>
<pre><code class="bash">$ docker run --rm --privileged -p 5050:5050 -p 5051:5051 -p 8080:8080 mesos/mesos-mini
</code></pre>
<p>It will launch one Mesos master, one Mesos agent, and one example framework (Marathon) in the Docker container.</p>
<p>You should be able to access Mesos master UI at <code>http://localhost:5050</code>.
Similarly, you can access Mesos agent at <code>http://localhost:5051</code>.
Marathon UI can be accessed at <code>http://localhost:8080</code>.</p>
<p>You should be able to launch containers in the local Mesos cluster using Marathon like the following:</p>
<pre><code class="bash">$ cat app.json
{
"id": "test",
"cmd": "sleep 1000",
"cpus": 1,
"mem": 128,
"disk": 0,
"instances": 1,
"container": {
"docker": {
"image": "alpine"
},
"type": "DOCKER"
},
"networks": [
{
"mode": "host"
}
]
}
$ curl -X POST -d @app.json -H "Content-type: application/json" http://localhost:8080/v2/apps
</code></pre>
<p>To stop the local Mesos cluster, please use <code>docker stop</code>.
All artifacts associated with the local Mesos cluster will be cleaned up when the Docker container stops.</p>
<p>The following Docker image tags are maintained:</p>
<ul>
<li><code>master</code>: The latest master branch HEAD.</li>
<li><code><RELEASE_BRANCH></code>: The latest release branch HEAD (e.g., <code>1.7.x</code>).</li>
<li><code>master-<DATE></code>: The snapshot builds for master branch (e.g., <code>master-2018-11-19</code>).</li>
<li><code><RELEASE_BRANCH>-<DATE></code>: The snapshot builds for release branch (e.g., <code>1.7.x-2018-11-19</code>).</li>
</ul>
<p>Note that there is no support for release branches earlier than <code>1.7.x</code>.
All future release branches will be supported.</p>
<h1>How is it done?</h1>
<h2>Manage multiple services</h2>
<p>We use <code>systemd</code> to manage multiple daemons in the Mesos Mini Docker container.
As a result, you can use the following command to restart the Mesos master in the Mesos Mini Docker container:</p>
<pre><code class="bash">$ docker exec <CONTAINER_ID> systemctl restart mesos-master
</code></pre>
<p>Similarly, you can use that to restart other services (e.g., Mesos agent or Marathon).
This is very useful for those end-to-end integration tests that want to simulate Mesos master failover.</p>
<h2>Docker Containerizer</h2>
<p>One of the goals of Mesos Mini is to mimic production settings as much as possible.</p>
<p>To allow frameworks to launch Docker containers, we embed a Docker Daemon (i.e., Docker in Docker) in the Mesos Mini Docker container.
For instance, to view all Docker containers in the Mesos cluster, use the following command on your host:</p>
<pre><code class="bash">$ docker exec <CONTAINER_ID> docker ps
</code></pre>
<p>The cgroup root for the embedded Docker Daemon has been configured so that the cgroups for the nested Docker containers are properly nested within the Mesos Mini Docker container.
This ensures that no cgroups traces will be left in the system when the Mesos Mini Docker container finishes.</p>
<h2>Mesos Containerizer (UCR)</h2>
<p>For Mesos Containerizer (UCR), we turn on most of the <a href="https://github.com/apache/mesos/docs/mesos-containerizer.md">isolators</a> that are typically turned on in production environments.
Similar to Docker daemon, we need to do a few tweaking on cgroups in Mesos Mini Docker container to make sure it does not leave any traces when Mesos Mini Docker container terminates.</p>
<p>For each cgroup subsystem, Docker does a bind mount from the current cgroup to the root of the cgroup subsystem.
For instance:</p>
<pre><code class="bash">/sys/fs/cgroup/memory/docker/<cid> -> /sys/fs/cgroup/memory
</code></pre>
<p>This will confuse Mesos agent and UCR because it relies on proc file <code>/proc/<pid>/cgroup</code> to determine the cgroups of a given process, and this proc file is not affected by the bind mount of the cgroups.</p>
<p>To workaround that, we perform the following steps for each cgroup subsystems when bootstrapping the Mesos Mini Docker container to recreates the cgroups layout as if it were on the host.</p>
<pre><code class="bash">$ mkdir -p /sys/fs/cgroup/memory/docker/<cid>
$ mount --bind /sys/fs/cgroup/memory /sys/fs/cgroup/memory/docker/<cid>
</code></pre>
<p>And then set Mesos agent <code>--cgroups_root</code> flag to <code>docker/<cid></code>.</p>
<h1>Maintenance</h1>
<p>The <a href="https://github.com/apache/mesos/tree/master/support/mesos-mini">build scripts</a> for Mesos Mini is hosted in Mesos repository.
The <a href="https://builds.apache.org/view/M-R/view/Mesos/job/Docker/job/Mini/">Mesos Docker Mini Jenkins CI</a> has been setup to automatically push daily snapshot builds to Docker hub for supported release branches as well as the master branch.</p>
<p>For any bug fix or new features, please follow the <a href="http://mesos.apache.org/community/#contribute-a-patch">Apache Mesos contribution guide</a>.</p>
<h2>Mesos CentOS Docker Image</h2>
<p>In some scenarios, some users might prefer having a Docker image that only has Mesos installed.
To enable that, we also built a <code>mesos/mesos-centos</code> Docker image.
The tags are similar to those of Mesos Mini.
In fact, <code>mesos/mesos-mini</code> uses <code>mesos/mesos-centos</code> as its base image.</p>
<p>The <a href="https://github.com/apache/mesos/tree/master/support/packaging/centos/build-docker-centos.sh">build scripts</a> for Mesos CentOS Docker image is also hosted in Mesos repository.
The <a href="https://builds.apache.org/view/M-R/view/Mesos/job/Docker/job/CentOS/">Mesos Docker CentOS Jenkins CI</a> has been setup to automatically push daily snapshot builds to Docker hub for supported release branches as well as the master branch.</p>
<h1>Current limitations</h1>
<ul>
<li>Only one Mesos agent can be launched in the Mesos Mini Docker container.</li>
<li>Marathon uses in-memory storage instead of ZK.</li>
<li>SSL is not enabled.</li>
</ul>
http://mesos.apache.org/blog/mesos-1-7-0-performance-improvements/
Performance Improvements in Mesos 1.7.0
2018-10-08T00:00:00+00:00Benjamin Mahler
<p><strong>Scalability and performance are key features for Mesos. Some users of Mesos already run production clusters that consist of many tens of thousands of nodes.</strong> However, there remains a lot of room for improvement across a variety of areas of the system.</p>
<p>The Mesos community has been working hard over the past few months to address several performance issues that have been affecting users. The following are some of the key performance improvements included in Mesos 1.7.0:</p>
<ul>
<li><strong>Master <code>/state</code> endpoint:</strong> Adopted <a href="http://rapidjson.org/">RapidJSON</a> and reduced copying for a 2.3x throughput improvement due to a ~55% decrease in latency (<a href="https://issues.apache.org/jira/browse/MESOS-9092">MESOS-9092</a>). Also, added parallel processing of <code>/state</code> requests to reduce master backlogging / interference under high request load (<a href="https://issues.apache.org/jira/browse/MESOS-9122">MESOS-9122</a>).</li>
<li><strong>Allocator:</strong> in 1.7.1 (these patches did not make 1.7.0 and were backported to 1.7.x), allocation cycle time was reduced. Some benchmarks show an 80% reduction. This, together with the reduced master backlogging from <code>/state</code> improvements, substantially reduces the end-to-end offer cycling time between Mesos and schedulers.</li>
<li><strong>Agent <code>/containers</code> endpoint:</strong> Fixed a performance issue that caused high latency / cpu consumption when there are many containers on the agent (<a href="https://issues.apache.org/jira/browse/MESOS-8418">MESOS-8418</a>).</li>
<li><strong>Agent container launching performance improvements</strong>: Some initial benchmarking shows a ~2x throughput improvement for both launch and destroy operations.</li>
</ul>
<p>Before we dive into the details of these performance improvements, I would like to recognize and thank the following contributors:</p>
<ul>
<li><strong>Alex Rukletsov</strong> and <strong>Benno Evers</strong>: for working on the parallel serving of master <code>/state</code> and providing benchmarking data.</li>
<li><strong>Meng Zhu</strong>: for authoring patches to help improve the allocator performance.</li>
<li><strong>Stéphane Cottin</strong> and <strong>Stephan Erb</strong>: for reporting the <code>/containers</code> endpoint performance issue and providing performance data.</li>
<li><strong>Jie Yu</strong>: for working on the container launching benchmarks and performance improvements.</li>
</ul>
<h2>Master <code>/state</code> Endpoint</h2>
<p>The <code>/state</code> endpoint of the master returns the full cluster state and is frequently polled by tooling (e.g. DNS / service discovery systems, backup systems, etc). We focused on improving performance of this endpoint as it is rather expensive and is common performance pain point for users.</p>
<h3>RapidJSON</h3>
<p>In Mesos we perform JSON serialization by directly going from C++ objects to serialized JSON via an internal library called <a href="https://github.com/apache/mesos/blob/1.6.0/3rdparty/stout/include/stout/jsonify.hpp">jsonify</a>. This library had some performance bottlenecks, primarily in the use of <code>std::ostream</code> for serialization:</p>
<ul>
<li>See <a href="https://groups.google.com/a/isocpp.org/forum/#!msg/std-proposals/bMzBAHgb5_o/C80lZHUwp5QJ">here</a> for a discussion of its performance issues with strings.</li>
<li>See <a href="https://github.com/miloyip/itoa-benchmark/tree/1f2b870c097d9444eec8e5c057b603a490e3d7ec#results">here</a> and <a href="https://github.com/miloyip/dtoa-benchmark/tree/c4020c62754950d38a1aaaed2975b05b441d1e7d#results">here</a> for integer-to-string and double-to-string performance comparisons against <code>std::ostream</code>.</li>
</ul>
<p>We found that RapidJSON has a performance focused approach that addresses these issues:</p>
<ul>
<li>Like <code>jsonify</code>, it also supports directly serializing from C++ without converting through intermediate JSON objects (via a <code>Writer</code> interface).</li>
<li>It eschews <code>std::ostream</code> (although it introduced support for it along with a <a href="http://rapidjson.org/md_doc_stream.html#iostreamWrapper">significant performance caveat</a>).</li>
<li>It performs fast integer-to-string and double-to-string conversions (see performance comparison linked above).</li>
</ul>
<p>After adapting <code>jsonify</code> to use RapidJSON and eliminating some additional <code>mesos::Resource</code> copying, we ran the master state query benchmark. This benchmark builds up a large simulated cluster in the master and times the end-to-end response time from a client’s perspective:</p>
<p><img src="/assets/img/blog/1.7-performance-improvements-rapidjson.png" alt="1.7 RapidJSON" /></p>
<p>This is a box plot, where the box indicates the range of the 1st and 3rd quartiles, and the lines extend to the minimum and maximum values. The results above showed a reduction in the client’s end-to-end time to receive the response from approximately 7 seconds down to just over 3 seconds when both rapidjson and the <code>mesos::Resource</code> copy elimination are applied. An 55% decrease in latency, which yields a 2.3x throughput improvement of state serving.</p>
<h3>Parallel Serving</h3>
<p>In Mesos, we use an asynchronous programming model based on actors and futures (see <a href="https://github.com/apache/mesos/tree/master/3rdparty/libprocess">libprocess</a>). Each actor in the system operates as an HTTP server in the sense that it can set up HTTP routes and respond to requests. The master actor hosts the majority of the v0 master endpoints, including <code>/state</code>. In an actor-based model, each actor has a queue of events and processes those events in serial (without parallelism). As a result, actors can be overwhelmed if there are too many expensive events in their queue. HTTP requests to the <code>/state</code> endpoint are an example of an expensive event, especially for larger clusters. If multiple clients are polling the <code>/state</code> endpoint, responsiveness of the master can degrade significantly.</p>
<p>In order to improve the ability to serve multiple clients of <code>/state</code>, we introduced parallel serving of the <code>/state</code> endpoint via a batching technique (see <a href="https://issues.apache.org/jira/browse/MESOS-9122">MESOS-9122</a>). This was possible since <code>/state</code> is read-only against the master actor, and we accomplish this by spawning other worker actors and blocking the master until they complete (see <a href="https://issues.apache.org/jira/browse/MESOS-8587">MESOS-8587</a> for potential library generalizations of this technique).</p>
<p>A benchmark was implemented that polls the master’s <code>/state</code> endpoint concurrently from multiple clients and measures the observed response times across 1.6.0 and 1.7.0:</p>
<p><img src="/assets/img/blog/1.7-performance-improvements-parallel-state.png" alt="1.7 Parallel state serving benchmark" /></p>
<p>The benchmark demonstrates a marked improvement in the response times as the number of clients polling the /state endpoint grows. These numbers were obtained using an optimized build on a machine with 4 x 2.9Ghz CPUs, and LIBPROCESS_NUM_WORKER_THREADS was set to 24. A virtual cluster was created with 100 agents, 10 running and 10 completed frameworks with 10 tasks each, on each agent. Every client polls the <code>/state</code> endpoint 50 times. Small dots denote raw measured response times, big dots denote the arithmetic mean. The top and bottom 2% of raw times were removed to filter outliers.</p>
<h2>Allocation Cycle Time</h2>
<p>Several users reported that the master’s resource allocator was taking a long time to perform allocation cycles on larger clusters (e.g. high agent / framework counts). We investigated this issue and found that the main scalability limitation was due to excessive re-computation of the DRF ordering of roles / frameworks (see <a href="https://issues.apache.org/jira/browse/MESOS-9249">MESOS-9249</a> and <a href="https://issues.apache.org/jira/browse/MESOS-9239">MESOS-9239</a> for details.</p>
<p>We ran an existing micro-benchmark of the allocator that creates clusters with a configurable number of agents and frameworks:</p>
<p><img src="/assets/img/blog/1.7-performance-improvements-allocation-cycle.png" alt="1.7 Allocation Cycle Benchmark" /></p>
<p>The results show an ~80% reduction in allocation cycle time in 1.7.1 for this particular setup (all frameworks in a single role, no filtering). Since this is a substantial improvement to a long-standing pain point for large scale users, we backported the changes to 1.7.1 since they are not included in 1.7.0.</p>
<p>Future work is underway to improve the allocation cycle performance when quota is enabled (see: <a href="https://issues.apache.org/jira/browse/MESOS-9087">MESOS-9087</a>).</p>
<h2>Agent <code>/containers</code> Endpoint</h2>
<p>Reported in <a href="https://issues.apache.org/jira/browse/MESOS-8418">MESOS-8418</a>, during the collection of container resource consumption metrics, there are many reads of <code>/proc/mounts</code> being performed. The system mount table will be large and expensive to read if there are a lot of containers running on the agent using their own root filesystems. This was only incidentally being done as a result of some cgroup related verification code performed before reading a cgroup file. Since this code was only in place to provide better error messages, it was removed.</p>
<p>Stephen Erb provided the following graphs that show the impact of deploying the change. First, we can see the tasks (i.e. containers) per agent:</p>
<p><img src="/assets/img/blog/1.7-performance-improvements-containers-endpoint-tasks.png" alt="1.7 containers endpoint task counts" /></p>
<p>The agent with the most tasks has ~150 containers, the median and average are both around 50 containers. The following graph provided by Stephan Erb shows the latency of the <code>/containers</code> endpoint before and after deploying the fix on the same cluster:</p>
<p><img src="/assets/img/blog/1.7-performance-improvements-containers-endpoint-latency.png" alt="1.7 containers endpoint latency" /></p>
<p>Prior to the change, the agent with the worst <code>/containers</code> latency took between 5-10 seconds to respond, and the median latency across agents was around 1 second. After the change, all of the agents have sub-second <code>/containers</code> latency.</p>
<h1>Agent Container Launching</h1>
<p>From the user reports originally in <a href="https://issues.apache.org/jira/browse/MESOS-8418">MESOS-8418</a>, we identified that the container launching throughput would suffer from the same issue of expensive <code>/proc/mounts</code> reads shown above in the <code>/containers</code> endpoint improvements. See <a href="https://issues.apache.org/jira/browse/MESOS-9081">MESOS-9081</a>.</p>
<p>To remedy this, we moved the cgroups verification code to the call-sites. Since the cgroup just needs to be verified once during the bootstrapping agent bootstrap, this optimization significantly reduces the overhead of launching and destroying containers.</p>
<p>A preliminary benchmark shows that the container launch / destroy throughput gained a 2x throughput improvement thanks to a 50% reduction in latency. This test uses an docker image based on the host OS image of the machine it’s running on:</p>
<p><img src="/assets/img/blog/1.7-performance-improvements-container-launch.png" alt="1.7 container launch" /></p>
<p>In this particular benchmark (see <a href="https://reviews.apache.org/r/68266/">reviews.apache.org/r/68266/</a>), a single agent is able to launch 1000 containers in about 30 seconds, destroying those 1000 containers in just over 20 seconds. This numbers were obtained on a server with 2 x Intel® Xeon® CPU E5-2658 v3.</p>
<h2>Performance Working Group Roadmap</h2>
<p>The backlog of performance worked in tracked in JIRA, see <a href="https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=238&useStoredSettings=true">here</a>. Any ticket with the <code>performance</code> label is picked up by this JIRA board.</p>
<p>If you are a user and would like to suggest some areas for performance improvement, please let us know by emailing <code>dev@apache.mesos.org</code> and we would be happy to help!</p>
http://mesos.apache.org/blog/mesos-1-7-0-released/
Mesos 1.7: Improvements to Performance, Containerization and Running Multiple Frameworks
2018-09-24T00:00:00+00:00Chun-Hung Hsiao & Gastón Kleiman
<p>We are excited to announce that Apache Mesos 1.7.0 is now available for <a href="/downloads">download</a>. Please take a look at what’s new in this release!</p>
<h1>New Features and Improvements</h1>
<h2>Performance</h2>
<p>In Mesos 1.7, we put a lot of emphasis on performance. Thanks to the community’s hard work, several performance issues that have been affecting users are significantly improved:</p>
<p><img src="/assets/img/blog/mesos-1-7-0-performance.png" alt="Mesos 1.7 performance improvements" /></p>
<ul>
<li>The master <code>/state</code> endpoint is now about twice as fast as before, and its processing is parallelized to provide higher throughput. As a result, Mesos UI and all tooling that consumes the <code>/state</code> endpoint are more responsive.</li>
<li><a href="http://mesos.apache.org/documentation/latest/mesos-containerizer/">Mesos containerizer</a> has been a high-performance alternative to the Docker runtime, and now it’s even faster: launch and destroy throughput has increased by 2x in this release.</li>
<li>The agent <code>/containers</code> endpoint no longer leads to high CPU consumption and thus is much more responsive. It now scales well to hundreds of containers running on the agent.</li>
<li>We made several optimizations to the resource allocator so the allocation cycle is now 20% faster on a 1,000-node cluster, and more significant improvements in this area are in progress for the 1.8 release.</li>
</ul>
<p>In addition, there are also a variety of other minor performance improvements across the system. A separate blog post that describes the 1.7.0 performance improvements in-depth will soon be published, so stay tuned!</p>
<h2>Containerization</h2>
<p>As usual, the 1.7.0 release includes several bug fixes to improve stability and reliability for users, as well as several new features and improvements:</p>
<ul>
<li>New <code>cgroups/all</code> agent isolation option: This option enables the cgroups isolator to detect supported cgroup subsystems automatically, so users no longer need to manually choose which cgroup subsystems to enable.</li>
<li>Better cgroupfs isolation: Containers launched by <a href="http://mesos.apache.org/documentation/latest/mesos-containerizer/">Mesos Containerizer</a> can now only see their own cgroups within <code>/sys/fs/cgroup</code>, and thus no container is allowed to see the cgroup information of other containers.</li>
<li>New <code>linux/devices</code> isolator: This isolator automatically populates containers with devices that have been whitelisted in the <code>--allowed_devices</code> agent flag.</li>
<li>Fetching Docker image tarballs from HDFS: Now users can specify an <code>hdfs://</code> URI in the <code>--docker_registry</code> agent flag to use HDFS in lieu of a Docker registry.</li>
<li>Better container network statistics: The built-in Container Network Interface (CNI) isolator now exposes more statistics to make users easier to monitor the network usage of each container.</li>
<li>Better image pulling metrics: The new <a href="http://mesos.apache.org/documentation/latest/monitoring/#containerizers">containerizer metrics</a> enable users to monitor the latency distribution for image pulling.</li>
</ul>
<h2>Framework Metrics</h2>
<p>To improve the user experience of operators when monitoring and diagnosing a cluster, we have added a variety of new metrics in Mesos 1.7 to provide greater visibility into the behavior of individual frameworks in the cluster.</p>
<p>These metrics are added on a per-framework basis, so that new metric keys appear as new frameworks register with the master. Example metrics include: the numbers of calls received, number of events sent, offer-related counters (how many were sent/accepted/declined), counts of the various offer operations performed, and current task state totals for the framework. Please see the <a href="http://mesos.apache.org/documentation/latest/monitoring/#frameworks">metrics documentation</a> for more details.</p>
<h2>Multi-Framework Workloads</h2>
<p>The following improvements, combined with the allocator and metrics improvements mentioned above, make the experience of running multiple frameworks significantly better in Mesos 1.7:</p>
<ul>
<li>We have improved the <a href="http://mesos.apache.org/documentation/latest/app-framework-development-guide/">framework development guide</a> with guidelines on how to implement frameworks that are well-behaved in a multi-framework cluster.</li>
<li>A new weighted random sorter was also added as an alternative to the existing DRF sorter, allowing users that don’t need DRF behavior to opt-out in favor of allocating to roles using a weighted uniform distribution.</li>
</ul>
<h1>Upgrade</h1>
<p>In most cases, upgrades from Mesos 1.6.0 to Mesos 1.7.0 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.7.0.</p>
<h1>Community</h1>
<p>Inspired by the work that went into this release? Want to get involved? Have feedback? We’d love to hear from you! Join a <a href="http://mesos.apache.org/community/#working-groups">working group</a> or start a conversation in the <a href="http://mesos.apache.org/community/">community</a>!</p>
<h1>Thank you!</h1>
<p>Thanks to the 50 contributors who made Mesos 1.7.0 possible:</p>
<p>Akash Gupta, Alexander Rojas, Alexander Rukletsov, Andrei Budnik, Andrew Schwartzmeyer, Armand Grillet, Benjamin Bannier, Benjamin Mahler, Benno Evers, Chen Runcong, Chun-Hung Hsiao, Clement Michaud, Dario Rexin, Eric Chung, Gary Yao, Gastón Kleiman, Gilbert Song, Greg Mann, Harold Dost, Ilya Pronin, James Peach, Jan Schlicht, Jiang Yan Xu, Jie Yu, John Kordich, Joseph Wu, Judith Malnick, Kapil Arya, Kevin Klues, Kjetil Joergensen, Meng Zhu, Pavi Sandhu, Qian Zhang, Radhika Jandhyala, Sergey Urbanovich, Tarun Gupta Akirala, Thodoris Zois, Till Toenshoff, Vinod Kone, Xudong Ni, Zhitao Li, Bin Zheng, Cai Shuhua, Cui Dt, He Yi Hua, janisz, LongFei Niu, Wei Xiao, Xiang Chaosheng, Zois</p>
http://mesos.apache.org/blog/mesoscon-2018-cfp-is-now-closed/
MesosCon 2018 CFP is now closed!
2018-09-05T00:00:00+00:00Andrew Schwartzmeyer
<p>Thank you everyone for submitting proposals, our Call For Proposals is now closed.</p>
<p>We would like to communicate an important change: MesosCon 2018 will instead be held in the San Francisco area on November 5th-7th, 2018. The dates remain unchanged, but it will not be held in New York City as previously announced.</p>
<p>After much discussion, the majority of the community found that the Bay Area would be easier to attend. We apologize for this last minute decision and any inconvenience that it might cause you. We are trying to have a small satellite event in New York City, and are also trying to allow for remote participation where feasible.</p>
<h2>CFP Decision Timeline</h2>
<p>Thank you again for everyone who submitted talks. We hope to inform each submitter by the end of September, and to announce the final format and venue at this time.</p>
<h2>Sponsor MesosCon</h2>
<p>MesosCon is a community run conference and your support is needed. Sponsoring this event will provide a great opportunity for your organization to further engage with the Mesos community. <a href="mailto:sponsor@mesoscon2018.org">Contact us</a> to learn more.</p>
<p>Help us make #MesosCon 2018 a big success by spreading the word!</p>
<p>Hope to see you at this year’s MesosCon!</p>
http://mesos.apache.org/blog/mesos-1.4.2-released
Apache Mesos 1.4.2 Released
2018-08-20T00:00:00+00:00Kapil Arya
<p>The latest Mesos 1.4.x release, 1.4.2, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes important bug fixes and improvements on top of 1.4.2. It is recommended to use this version if you are considering using Mesos 1.4. More specifically, this release includes the following notable fixes:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4527">MESOS-4527</a> - Roles can exceed limit allocation via reservations.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7099">MESOS-7099</a> - Quota can be exceeded due to coarse-grained offer technique.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8293">MESOS-8293</a> - Reservation may not be allocated when the role has no quota.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8339">MESOS-8339</a> - Quota headroom may be insufficiently held when role has more reservation than quota.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8352">MESOS-8352</a> - Resources may get over allocated to some roles while fail to meet the quota of other roles.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8488">MESOS-8488</a> - Docker bug can cause unkillable tasks.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8604">MESOS-8604</a> - Quota headroom tracking may be incorrect in the presence of hierarchical reservation.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8830">MESOS-8830</a> - Agent gc on old slave sandboxes could empty persistent volume data</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8904">MESOS-8904</a> - Master crash when removing quota.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8935">MESOS-8935</a> - Quota limit “chopping” can lead to cpu-only and memory-only offers.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8936">MESOS-8936</a> - Implement a Random Sorter for offer allocations.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8942">MESOS-8942</a> - Master streaming API does not send (health) check updates for tasks.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8987">MESOS-8987</a> - Master asks agent to shutdown upon auth errors.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9125">MESOS-9125</a> - Port mapper CNI plugin might fail with “Resource temporarily unavailable”</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9127">MESOS-9127</a> - Port mapper CNI plugin might deadlock iptables on the agent.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.4.2">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.4.1 cluster to Mesos 1.4.2 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.4.1 from 1.0.x, 1.2.x, or 1.3.x.</p>
<h3>Try it out</h3>
<p>Please try out this release and let us know what you think. If you run into any issues, let us know on the <a href="https://mesos.apache.org/community">user mailing list and/or Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 24 contributors who made 1.4.2 possible:</p>
<p>Akash Gupta, Alexander Rojas, Alexander Rukletsov, Andrei Budnik, Armand Grillet, Benjamin Bannier, Benjamin Hindman, Benjamin Mahler, Benno Evers, Bin Zheng, Chun-Hung Hsiao, Gastón Kleiman, Gilbert Song, Greg Mann, He Yi Hua, James Peach, Jason Lai, Jie Yu, Kapil Arya, Kjetil Joergensen, Meng Zhu, Michael Park, Qian Zhang, Zhitao Li</p>
http://mesos.apache.org/blog/mesoscon-2018-cfp-is-now-open/
MesosCon 2018 CFP is now open!
2018-07-30T00:00:00+00:00Gastón Kleiman
<p>MesosCon is an annual conference organized by the Apache Mesos community, bringing together users and developers to share and learn about the project and its growing ecosystem.</p>
<p>We are excited to announce that MesosCon 2018 will be held in San Francisco on November 5th-7th, 2018.</p>
<p>Talk submissions and sponsorship opportunities are now open for the conference!</p>
<h2>Speak at MesosCon</h2>
<p>Several formats are being accepted for speaking proposals, including: Presentations, Panels, Keynotes, and Lightning Talks. Submissions are being accepted through <strong>August 27, 2018</strong>.</p>
<p>You can take a look at the <a href="https://www.youtube.com/playlist?list=PLbzoR-pLrL6qAEnkhkh5tGI6oX_xXD3X4">MesosCon 2017 list of talks</a> for ideas.</p>
<p>If you are unsure about your proposal, or want some feedback or advice in general, please do not hesitate to reach out to the <a href="https://mail-archives.apache.org/mod_mbox/mesos-dev">mesos-dev</a> mailing list. We will be happy to help out! Further details are available on the <a href="https://mesoscon2018.org">CFP website</a>.</p>
<h2>Sponsor MesosCon</h2>
<p>MesosCon is a community run conference and your support is needed. Sponsoring this event will provide a great opportunity for your organization to further engage with the Mesos community. <a href="mailto:sponsor@mesoscon2018.org">Contact us</a> to learn more.</p>
<p>Help us make #MesosCon 2018 a big success by spreading the word and submitting your proposal!</p>
<p>Hope to see you at this year’s MesosCon!</p>
<p><strong>Edit:</strong> This post previously said New York City, but this was changed to San Francisco.</p>
http://mesos.apache.org/blog/mesos-1-6-1-released/
Apache Mesos 1.6.1 Released
2018-07-25T00:00:00+00:00Greg Mann
<p>Apache Mesos 1.6.1 is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes important bug fixes and improvements to 1.6.0; if you’re considering using Mesos 1.6, it’s recommended that you use 1.6.1. The following notable fixes are included:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-3790">MESOS-3790</a> - ZooKeeper connection should retry on <code>EAI_NONAME</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8830">MESOS-8830</a> - Agent gc on old slave sandboxes could empty persistent volume data</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8871">MESOS-8871</a> - Agent may fail to recover if the agent dies before image store cache checkpointed.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8904">MESOS-8904</a> - Master crash when removing quota.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8936">MESOS-8936</a> - Implement a Random Sorter for offer allocations.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8945">MESOS-8945</a> - Master check failure due to CHECK_SOME(providerId).</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8963">MESOS-8963</a> - Executor crash trying to print container ID.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8980">MESOS-8980</a> - mesos-slave can deadlock with docker pull.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8986">MESOS-8986</a> - <code>slave.available()</code> in the allocator is expensive and drags down allocation performance.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8987">MESOS-8987</a> - Master asks agent to shutdown upon auth errors.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9002">MESOS-9002</a> - GCC 8.1 build failure in os::Fork::Tree.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9024">MESOS-9024</a> - Mesos master segfaults with stack overflow under load.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-9025">MESOS-9025</a> - The container which joins CNI network and has checkpoint enabled will be mistakenly destroyed by agent.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.6.1">CHANGELOG</a>.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.6.0 cluster to Mesos 1.6.1 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.6.1 from other versions.</p>
<h3>Try it out</h3>
<p>Please try out this release and let us know what you think. If you run into any issues, let us know on the <a href="https://mesos.apache.org/community">user mailing list and/or Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 18 contributors who made 1.6.1 possible:</p>
<p>Alexander Rojas, Alexander Rukletsov, Andrew Schwartzmeyer, Armand Grillet, Benjamin Bannier, Benjamin Mahler, Benno Evers, bin zheng, Chun-Hung Hsiao, Gastón Kleiman, Gilbert Song, Greg Mann, he yi hua, James Peach, Jie Yu, Kjetil Joergensen, Meng Zhu, Zhitao Li</p>
http://mesos.apache.org/blog/mesos-1-5-1-released/
Apache Mesos 1.5.1 Released
2018-05-31T00:00:00+00:00Gilbert Song
<p>The latest Mesos 1.5.x release, 1.5.1, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes important bug fixes and improvements on top of 1.5.0. It is recommended to use this version if you are considering using Mesos 1.5. More specifically, this release includes the following:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1720">MESOS-1720</a> - Slave should send exited executor message when the executor is never launched.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7742">MESOS-7742</a> - Race conditions in IOSwitchboard: listening on unix socket and premature closing of the connection.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8125">MESOS-8125</a> - Agent should properly handle recovering an executor when its pid is reused.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8411">MESOS-8411</a> - Killing a queued task can lead to the command executor never terminating.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8416">MESOS-8416</a> - CHECK failure if trying to recover nested containers but the framework checkpointing is not enabled.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8468">MESOS-8468</a> - <code>LAUNCH_GROUP</code> failure tears down the default executor.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8488">MESOS-8488</a> - Docker bug can cause unkillable tasks.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8510">MESOS-8510</a> - URI disk profile adaptor does not consider plugin type for a profile.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8536">MESOS-8536</a> - Pending offer operations on resource provider resources not properly accounted for in allocator.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8550">MESOS-8550</a> - Bug in <code>Master::detected()</code> leads to coredump in <code>MasterZooKeeperTest.MasterInfoAddress</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8552">MESOS-8552</a> - CGROUPS_ROOT_PidNamespaceForward and CGROUPS_ROOT_PidNamespaceBackward tests fail.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8565">MESOS-8565</a> - Persistent volumes are not visible in Mesos UI when launching a pod using default executor.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8569">MESOS-8569</a> - Allow newline characters when decoding base64 strings in stout.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8574">MESOS-8574</a> - Docker executor makes no progress when ‘docker inspect’ hangs.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8575">MESOS-8575</a> - Improve discard handling for ‘Docker::stop’ and ‘Docker::pull’.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8576">MESOS-8576</a> - Improve discard handling of ‘Docker::inspect()’.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8577">MESOS-8577</a> - Destroy nested container if <code>LAUNCH_NESTED_CONTAINER_SESSION</code> fails.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8594">MESOS-8594</a> - Mesos master stack overflow in libprocess socket send loop.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8598">MESOS-8598</a> - Allow empty resource provider selector in <code>UriDiskProfileAdaptor</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8601">MESOS-8601</a> - Master crashes during slave reregistration after failover.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8604">MESOS-8604</a> - Quota headroom tracking may be incorrect in the presence of hierarchical reservation.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8605">MESOS-8605</a> - Terminal task status update will not send if ‘docker inspect’ is hung.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8619">MESOS-8619</a> - Docker on Windows uses <code>USERPROFILE</code> instead of <code>HOME</code> for credentials.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8624">MESOS-8624</a> - Valid tasks may be explicitly dropped by agent due to race conditions.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8631">MESOS-8631</a> - Agent should be able to start a task with every CPU on a Windows machine.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8641">MESOS-8641</a> - Event stream could send heartbeat before subscribed.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8646">MESOS-8646</a> - Agent should be able to resolve file names on open files.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8651">MESOS-8651</a> - Potential memory leaks in the <code>volume/sandbox_path</code> isolator.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8741">MESOS-8741</a> - <code>Add</code> to sequence will not run if it races with sequence destruction.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8742">MESOS-8742</a> - Agent resource provider config API calls should be idempotent.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8786">MESOS-8786</a> - CgroupIsolatorProcess accesses subsystem processes directly.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8787">MESOS-8787</a> - RP-related API should be experimental.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8876">MESOS-8876</a> - Normal exit of Docker container using rexray volume results in TASK_FAILED.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8881">MESOS-8881</a> - Enable epoll backend in libevent integration.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8885">MESOS-8885</a> - Disable libevent debug mode.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.5.1">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.5.0 cluster to Mesos 1.5.1 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.5.1 from 1.4.x, 1.3.x, or 1.2.x.</p>
<h3>Try it out</h3>
<p>Please try out this release and let us know what you think. If you run into any issues, let us know on the <a href="https://mesos.apache.org/community">user mailing list and/or Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 23 contributors who made 1.5.1 possible:</p>
<p>Akash Gupta, Alexander Rukletsov, Andrei Budnik, Andrew Schwartzmeyer, Benjamin Bannier, Benjamin Mahler, Benno Evers, Chun-Hung Hsiao, Gaston Kleiman, Gilbert Song, Greg Mann, Ilya Pronin, James Peach, Jason Lai, Jie Yu, John Kordich, Kapil Arya, Meng Zhu, Michael Park, Qian Zhang, Tomasz Janiszewski, Vinod Kone, Zhitao Li</p>
http://mesos.apache.org/blog/mesos-1-6-0-released/
Mesos 1.6: Persistent Volume, Scheduler API, and Docker Support Improvements
2018-05-17T00:00:00+00:00Greg Mann
<p>Great work everyone; Apache Mesos 1.6.0 was released on May 11!</p>
<h1>New Features and Improvements</h1>
<p>Mesos 1.6 includes improvements to persistent volumes, usability of the scheduler API, Docker daemon support, and memory allocation performance.</p>
<h2>Resizing Persistent Volumes</h2>
<p>Mesos 1.6 adds experimental support for the <a href="http://mesos.apache.org/documentation/latest/persistent-volume/">resizing of persistent volumes</a>. Two new operations, <a href="http://mesos.apache.org/documentation/latest/persistent-volume/#offer-operation-growvolume">GrowVolume</a> and <a href="http://mesos.apache.org/documentation/latest/persistent-volume/#offer-operation-shrinkvolume">ShrinkVolume</a>, can be used both via the <a href="http://mesos.apache.org/documentation/latest/scheduler-http-api/#accept">scheduler API</a> and via the <a href="http://mesos.apache.org/documentation/latest/operator-http-api/#grow_volume">operator API</a>.</p>
<p>Persistent volume resizing is currently limited to volumes created on default agent disk resources - in other words, CSI volumes are not yet supported.</p>
<h2>Offer Operation Feedback</h2>
<p>Mesos 1.6 introduces experimental support for offer operation status updates. Previously, only <code>Launch</code> and <code>LaunchGroup</code> operations received feedback via task status updates. Now, schedulers may opt to receive feedback on other offer operations via operation status updates by setting the <code>id</code> field in the operation. Similar to task status updates, when these operation updates have a unique <code>uuid</code> set, the scheduler must acknowledge them. This feature allows schedulers to ensure the reliability of offer operations like <code>Reserve</code> and <code>Create</code>; if such an operation fails and the scheduler has opted in to receive feedback, an operation status update will be sent and the scheduler can retry if appropriate.</p>
<p>Offer operation feedback is currently limited to operations on resources provided by a resource provider - in most cases, this means that only operations on CSI volumes are supported.</p>
<h2>Performance and Debugging</h2>
<p>Mesos 1.6 includes experimental support for the <a href="http://jemalloc.net">jemalloc</a> memory allocation library. In addition to a potential improvement in memory allocation performance, this enables simple memory profiling via command-line flags and environment variables outlined in the <a href="http://mesos.apache.org/documentation/latest/memory-profiling/">documentation</a>. We expect this feature to help drive future performance improvments in Mesos since it improves the ability of operators to understand how Mesos behaves in production environments.</p>
<h2>Docker Improvements</h2>
<p>In response to issues encountered by operators when using Mesos with the Docker container runtime, <a href="https://issues.apache.org/jira/browse/MESOS-8572">several improvements</a> were made to the Docker containerizer and executor to help them more gracefully handle situations in which the Docker daemon is unresponsive. Retry and timeout logic has been added in several key locations so that when the Docker daemon hangs for short periods of time, Mesos can recover and resume normal operation.</p>
<h2>Upgrade</h2>
<p>In most cases, upgrades from Mesos 1.5.0 to Mesos 1.6.0 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.6.0.</p>
<p>Support for the Container Storage Interface (CSI) version 0.2 has been added, and CSI v0.1 support is now deprecated.</p>
<p>When building Mesos with gRPC support, note that gRPC version 1.10 or later is now required.</p>
<h1>Community</h1>
<p>Inspired by the work that went into this release? Want to get involved? Have feedback? We’d love to hear from you! Join a <a href="http://mesos.apache.org/community/#working-groups">working group</a> or start a conversation in the <a href="http://mesos.apache.org/community/">community</a>!</p>
<h1>Thank you!</h1>
<p>Thanks to the 55 contributors who made Mesos 1.6.0 possible:</p>
<p>Adam B, Akash Gupta, Alexander Rojas, Alexander Rukletsov, Andrei Budnik, Andrew Schwartzmeyer, AntonDan, Armand Grillet, Benjamin Bannier, Benjamin Mahler, Benjamin Peterson, Benno Evers, Chun-Hung Hsiao, Clement Michaud, David Forsythe, Demitri Swan, EMumau, Eric Chung, Eric Mumau, Gaston Kleiman, Gilbert Song, Greg Mann, Harold Dost, Ilya Pronin, James Peach, Jan Schlicht, Jason Lai, Jeff Coffler, Jiang Yan Xu, Jie Yu, Joerg Schad, John Kordich, Joseph Wu, Judith Malnick, Kapil Arya, Marcin Owsiany, Megha Sharma, Meng Zhu, Michael Park, Murilo Pereira, Packt, Qian Zhang, Sachin Paryani, Sagar Patwardhan, Senthil Kumaran, Shiv Deepak, Shubham Kumar, Sofian Brabez, Till Toenshoff, Tomasz Janiszewski, Vinod Kone, Xudong Ni, Zhitao Li, longfei, miamipanther</p>
<p>Enjoy Apache Mesos 1.6!</p>
http://mesos.apache.org/blog/mesos-1-5-0-released/
Mesos 1.5: Storage, Performance, Resource Management, and Containerization improvements
2018-02-08T00:00:00+00:00Gilbert Song
<p>I’m pleased to announce that Apache Mesos 1.5.0 was released today!</p>
<h1>New Features and Improvements</h1>
<p>Mesos 1.5 includes significant improvements to resource management, storage and containerization. Container Storage Interface (CSI) support is new and experimental in Mesos 1.5, and many of the 1.5 features were added to support CSI. The new ability to reconfigure agents has made the operator experience better. Performance gains have made master failover 80-85% faster, and the v1 API performance has improved significantly. Resource management is more flexible with the new release. Last but not least, image garbage collection and better Windows support make it easier to run Mesos on different types of infrastructure, and to run different types of workloads on Mesos.</p>
<p>Read about all these improvements below, and enjoy Mesos 1.5.</p>
<h2>Container Storage Interface Support</h2>
<p>Mesos 1.5 adds experimental support for the <a href="https://github.com/container-storage-interface/spec">Container Storage Interface (CSI)</a>.CSI is a specification that defines a common set of APIs for all interactions between storage vendors and container orchestration platforms. It is the result of a close collaboration among representatives from the Mesos, <a href="https://kubernetes.io/">Kubernetes</a>, <a href="https://www.cloudfoundry.org/">Cloud Foundry</a>, and <a href="https://www.docker.com/">Docker</a> communities. The primary goal of CSI is to allow storage vendors to write one plugin that works with all container orchestration platforms.</p>
<p>Supporting CSI will help Mesos consistently fit into the larger storage ecosystem, resulting in better storage support in Mesos. Users will be able to use any storage system with Mesos using a consistent API. The out-of-tree plugin model of CSI decouples the release cycle of Mesos from that of the storage systems, making the integration itself more sustainable and maintainable.</p>
<p><img src="/assets/img/documentation/csi-architecture.png" alt="CSI Support in Mesos" /></p>
<p>The above figure shows the high-level architecture of how CSI is supported in Mesos. For more details, refer to this <a href="http://mesos.apache.org/documentation/latest/csi/">documentation</a>.</p>
<h2>Improved Operator Experiences</h2>
<h3>Agent Reconfiguration Policy</h3>
<p>Prior to Mesos 1.5, it was not possible to make any changes to the configuration of an agent without killing all tasks running on that agent and restarting with a new agent ID. Any change, for example, an additional attribute or a newly connected hard-drive, would be detected and abort the agent startup with a nasty-looking error:</p>
<pre><code>EXIT with status 1: Failed to perform recovery: Incompatible agent info detected.
------------------------------------------------------------
[...]
------------------------------------------------------------
To remedy this do as follows:
Step 1: rm -f /path/to/work_dir/meta/slaves/latest
This ensures agent doesn't recover old live executors.
Step 2: Restart the agent.
</code></pre>
<p>Starting with 1.5, operators can use the new agent command-line flag <code>--reconfiguration_policy</code> to configure the types of operations that should be allowed on agents, and which should lead to errors. When setting the value of this flag to <code>additive</code>, the agent will tolerate any changes that increase the amount of resources on the agent, as well as changes that add additional attributes or fault domains.</p>
<p>In the future, we hope to provide the additional setting <code>--reconfiguration_policy=any</code> to allow any changes to the agent configuration.</p>
<h2>Performance improvements</h2>
<p>This release includes improvements to master failover and v1 operator state query API performance. Master failover time-to-completion achieved a 450-600% improvement in throughput, which reduces the time-to-completion by 80-85%.</p>
<p><img src="/assets/img/documentation/1.3-1.5_master_failover_with_history.png" alt="1.3 - 1.5 Master Failover with Task History Graph" /></p>
<p>See <a href="http://mesos.apache.org/blog/performance-working-group-progress-report/">our blog post</a> for more details.</p>
<p>In addition, the performance of v1 operator state query API has also been greatly improved thanks to unnecessary copy eliminations. Specifically, v1 protobuf GetState call is 36% faster compared to v0. And, v1 JSON GetState performance has improved by more than four times.</p>
<p><img src="/assets/img/documentation/1.3-1.5_v1_json_state_query_latency.png" alt="1.3 - 1.5 v1 JSON State Query Latency" /></p>
<p>Detailed benchmark results can be found in <a href="https://docs.google.com/spreadsheets/d/1mN4OLEi7UdbLxv-Q3k3iMlmO6Gb2wx9X-gHRnDENSK0/edit#gid=1562845084">this spreadsheet</a>.</p>
<h2>Resource Management</h2>
<h3>Quota guarantee improvements</h3>
<p>Several quota related improvements were made in this release. Now Mesos does a better job ensuring that a role receives its quota and that a role does not exceed its quota, e.g.:</p>
<ul>
<li>Previously a role could “game” the quota system by amassing reservations that it leaves unused. This is now prevented by <a href="https://issues.apache.org/jira/browse/MESOS-4527">accounting for reservations</a> when allocating resources.</li>
<li>Resources are now <a href="https://issues.apache.org/jira/browse/MESOS-7099">allocated in a fine-grained manner</a> to prevent roles from exceeding their quota.</li>
<li>There was a <a href="https://issues.apache.org/jira/browse/MESOS-8293">bug</a> where roles without quota may not receive their reservations. This has been fixed.</li>
<li>When a role had more reservations than its quota, there was a <a href="https://issues.apache.org/jira/browse/MESOS-8339">bug</a> where an insufficient amount of quota headroom was held. This has been fixed.</li>
<li>When allocating resources to a role with quota, we previously also included resources that the role did not have the quota for on the agent. This made it possible to violate the quota guarantees of a different role. This has been fixed by <a href="https://issues.apache.org/jira/browse/MESOS-8352">taking into account the headroom</a> that is needed when allocating the resources.</li>
</ul>
<h3>Resource Provider abstraction</h3>
<p>To decouple resource and agent life cycles we have introduced an abstraction for <a href="https://issues.apache.org/jira/browse/MESOS-7235">resource providers</a>. This lays the groundwork to support dynamic changes to agent resources, integration of external resource providers, or management of cluster-wide resources. As part of the decoupling of resource responsibilities, we have introduced a new protocol to distribute operations on resources. This allows them to explicitly communicate success or failure, and enables reconciliation between cluster components. We have used the new resource provider abstraction to integrate agent-local storage resources interfaced with CSI.</p>
<h2>Containerization and Multi-Platform Support</h2>
<h3>Windows Support Improvements</h3>
<p>With the release of Mesos 1.5, Windows support has been vastly improved. Now when running tasks with the Mesos containerizer, resource limits can be enforced at the operating system level via job objects (<a href="https://issues-test.apache.org/jira/browse/MESOS-6690">MESOS-6690</a>). When the <a href="https://mesos.apache.org/documentation/latest/isolators/windows/#cpu-limits">windows/cpu</a> and <a href="https://mesos.apache.org/documentation/latest/isolators/windows/#memory-limits">windows/mem</a> isolators are used, a hard cap on the CPU cycles and virtual memory can be enforced. Additionally, the CPU and memory statistics are now correctly supported. The Mesos fetcher has been <a href="https://issues-test.apache.org/jira/browse/MESOS-8180">ported to Windows</a>, with support for automatically downloading files over TLS, and extracting ZIP archives. When configured for it, libprocess can now be built with <a href="https://issues-test.apache.org/jira/browse/MESOS-7992">OpenSSL support on Windows</a>, allowing TLS connections to be enforced within the cluster (note that this relies on building or installing OpenSSL externally, as it is not shipped with Windows). The first authentication method, CRAM-MD5, has been enabled on Windows, so that agents can authenticate with masters. Both HTTP and TCP <a href="https://issues-test.apache.org/jira/browse/MESOS-6709">health checks</a> are now available for tasks on Windows agents (for shell tasks at least; health checks for Docker containers are coming soon). The Mesos agent no longer has to be run as an administrator, as it is now possible to <a href="https://issues-test.apache.org/jira/browse/MESOS-7370">create symlinks as a normal user</a>, and this feature was integrated into the agent. Finally, while this was available in the previous release, <a href="https://issues-test.apache.org/jira/browse/MESOS-7371">native long path support for Windows</a> has been implemented, so that running the agent on Windows is now straightforward (with no more registry configurations).</p>
<h3>Container Image GC</h3>
<p>Mesos 1.5 has support for Container Image Garbage Collection. There are two ways to garbage collect unused image layers: automatically or manually. This feature helps users avoid unbounded disk space usage in the docker image store. Please see <a href="http://mesos.apache.org/documentation/latest/container-image/#garbage-collect-unused-container-images">Container Image GC Document</a> for more details.</p>
<h3>Standalone Container</h3>
<p>This release includes a set of new operator APIs for launching and managing a new primitive called a “Standalone Container”. A Standalone Container is similar to a container launched by a framework on the Mesos Agent, except that Standalone Containers are launched directly on the Mesos Agent by the operator. As a result, these containers do not use a Mesos Executor and have some other limitations (such as not being able to use reserved resources). At the same time, Standalone Containers are isolated like any other container and can use the same set of containerization features, like container images.</p>
<p>This feature is being released to support the <a href="http://mesos.apache.org/documentation/latest/csi/">CSI</a> workflow but the APIs are not restricted to this use case. This feature may potentially be used to implement <a href="https://issues.apache.org/jira/browse/MESOS-6595">daemon processes for agent</a> nodes in future. See <a href="https://issues.apache.org/jira/browse/MESOS-7302">MESOS-7302</a> for more details.</p>
<h2>Mesos Internals</h2>
<h3>Support gRPC Client</h3>
<p>This release includes a gRPC client wrapper. <a href="https://grpc.io/">gRPC</a> is an RPC mechanism that is becoming increasingly popular in the cloud-native space as an alternative to REST API. As a first step to integrating gRPC into Mesos, we have added this client wrapper in libprocess to support CSI. This feature is disabled by default, and can be enabled with the <code>--enable-grpc</code> (autotools) build flag on clusters that have been built with the <a href="https://www.openssl.org/">OpenSSL</a> library installed. The wrapper class maintains gRPC runtime data structures internally, and provides a simple interface to make asynchronous gRPC calls that are compatible to libprocess’ actor-based model. We have used this new feature to interface with CSI plugins. In addition, people can also use this new feature to create Mesos modules to talk to their own gRPC-based services.</p>
<h3>Replicated log improvements</h3>
<p>Mesos replicated log is used by frameworks (e.g. Apache Aurora) to persist their state across multiple replicas so it can survive failovers and restarts. Starting with 1.5 it is possible to perform eventually consistent reads from a non-leading VOTING replicated log replica. This makes possible to do additional work on non-leading framework replicas, such as offloading some reading from a leader to standbys, or reducing failover time by keeping in-memory storage represented by the replicated log “hot”.</p>
<h1>Upgrade</h1>
<p>Rolling upgrades from a Mesos 1.4.0 cluster to Mesos 1.5.0 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.5.0.</p>
<h1>Community</h1>
<p>Inspired by the work that went into this release? Want to get involved? Have feedback? We’d love to hear from you! Join a <a href="http://mesos.apache.org/community/#working-groups">working group</a> or start a conversation in the <a href="http://mesos.apache.org/community/">community</a>!</p>
<h1>Thank you!</h1>
<p>Thanks to the 59 contributors who made Mesos 1.5.0 possible:</p>
<p>Aaron Wood, Adam B, Adam Dangoor, Akash Gupta, Alastair Montgomery, Alexander Rojas, Alexander Rukletsov, Anand Mazumdar, Andrei Budnik, Andrew Schwartzmeyer, Andrey Dudin, Andy Pang, Anindya Sinha, Armand Grillet, Avinash sridharan, Benjamin Bannier, Benjamin Hindman, Benjamin Mahler, Benno Evers, Bob Eckert, Chun-Hung Hsiao, Cynthia Thomas, Deepak Goel, Dmitry Zhuk, Gaston Kleiman, Gastón Kleiman, Gilbert Song, Greg Mann, Ilya Pronin, Jack Yang, James DeFelice, James Peach, Jan Schlicht, Jason Lai, Jeff Coffler, Jiang Yan Xu, Jie Yu, Joerg Schad, John Kordich, Joseph Wu, Julien Pepy, Kapil Arya, Kevin Klues, Megha Sharma, Meng Zhu, Michael Park, Nathan Jackson, Neil Conway, Packt, Pranay Kanwar, Qian Zhang, Quinn Leng, Richard Shaw, ThodorisZois, Till Toenshoff, Tomas Barton, Tomasz Janiszewski, Vinod Kone, Zhitao Li</p>
<p>Enjoy Apache Mesos 1.5!</p>
http://mesos.apache.org/blog/storage-developments-in-apache-mesos/
Storage developments in Apache Mesos: Interview with Jie Yu
2017-12-12T00:00:00+00:00Greg Mann
<p>The <a href="https://github.com/container-storage-interface">Container Storage Interface</a> (CSI) is a community driven effort to standardize the way that containers connect to storage, so that any compliant container platform will be able to use storage from any compliant ecosystem vendor. I recently sat down with long-time Mesos committer Jie Yu, who is participating heavily in the development of CSI, to discuss the effort that he’s been leading to build CSI compatibility into Mesos.
<br>
<br>
<strong>Greg</strong>: Before we get started, could you tell us a bit about your history with the Apache Mesos project?
<br>
<strong>Jie</strong>: I’ve been working on Mesos for 4 years. I started at Twitter, and then came to Mesosphere 2 years ago. I’ve worked on many container-related things in the codebase. This includes - among other things - networking and, more recently, storage.</p>
<p><strong>Greg</strong>: Could you summarize, at a high level, the direction in which you’re currently taking Mesos storage support?
<br>
<strong>Jie</strong>: There are many storage vendors out there, and we don’t want to build all this vendor-specific code ourselves. Instead, we want a system that allows storage vendors to plug into Mesos using a consistent interface. Then we don’t need to maintain code for each vendor, which requires specific expertise.
Some other projects ask vendors to put their code into the main codebase. This can be painful because then the vendor’s release cycle is tied to the cycle of the project itself. This is hard to manage if you have many vendors. That’s why we decided to go with this “out-of-tree” solution for storage.</p>
<p><strong>Greg</strong>: What use cases are you most excited to enable?
<br>
<strong>Jie</strong>: Historically, we didn’t support global resources like EBS. We have limited support using <a href="https://docs.docker.com/engine/extend/plugins_volume/">DVDI</a>, but there are issues with this. I think external storage is something I’m really excited about with this work. Being able to plug in arbitrary vendors’ storage systems is exciting.
The other thing I’m excited about is that we have a solution that allows a vendor to provide just a single container image for their plugin, and we’ll take care of the rest of the work end-to-end. So it will be very simple for people to integrate their storage system with Mesos.</p>
<p><strong>Greg</strong>: When can we expect these features to become available?
<br>
<strong>Jie</strong>: The plan is to have local CSI storage support in Mesos 1.5, then external storage support in 1.6.</p>
<p><strong>Greg</strong>: How can community members get involved going forward?
<br>
<strong>Jie</strong>: Well, there are multiple avenues. For storage vendors, you can write a CSI plugin and test it with Mesos. For Mesos users, please try out this feature! And if you want to contribute to the coding, you can reach out on the <a href="mailto:dev@apache.mesos.org">mailing list</a>. Hopefully we can organize a storage working group in the near future once the feature lands.</p>
<p><strong>Greg</strong>: What’s next after Mesos 1.6?
<br>
<strong>Jie</strong>: If we can wrap up our plans for 1.6, then we’ll be in pretty good shape. The next thing is building a great storage ecosystem for Mesos. We can reach out to vendors and try to get them to integrate.</p>
<p><strong>Greg</strong>: Is there anything else you’d like to share with the community?
<br>
<strong>Jie</strong>: One thing to highlight is that while we’re doing storage work right now, the abstractions we’ve built for storage can be used for other purposes in Mesos - for example, they could be used for other <a href="https://issues.apache.org/jira/browse/MESOS-2728">global resources</a> like IP addresses. In the cloud, you could use this to manage things like AWS’s ENI.
In the future, we probably need a general interface to support arbitrary devices. The industry trend is the introduction of more and more specialized devices to support things like machine learning. We probably need a general interface for these types of devices so that Mesos can expose them in a unified way.
<br>
<br>
To learn more about CSI work in Mesos, you can dig into the design document <a href="https://docs.google.com/document/d/125YWqg_5BB5OY9a6M7LZcby5RSqBwo2PZzpVLuxYXh4/edit?usp=sharing">here</a>, and track the ongoing progress on <a href="https://issues.apache.org/jira/browse/MESOS-7235">Apache JIRA</a>. To contribute to the multi-platform effort to implement CSI, join the <a href="https://github.com/container-storage-interface/community#community">CSI Community</a>.</p>
http://mesos.apache.org/blog/performance-working-group-progress-report/
December 2017 Performance Working Group Progress Report
2017-12-07T00:00:00+00:00Benjamin Mahler
<p><strong>Scalability and performance are key features for Mesos. Some users of Mesos already run production clusters that consist of more than 35,000+ agents and 100,000+ active tasks.</strong> However, there remains a lot of room for improvement across a variety of areas of the system.</p>
<p>The performance working group was created in order to focus on some of these areas. The group’s charter is to improve scalability / throughput / latency across the system, and in order to measure our improvements and prevent performance regressions we will write benchmarks and automate them.</p>
<p>In the past few months, we’ve focused on making improvements to the following areas:</p>
<ul>
<li><strong>Master failover time-to-completion</strong>: Achieved a 450-600% improvement in throughput, which reduces the time-to-completion by 80-85%.</li>
<li><strong><a href="https://github.com/apache/mesos/tree/master/3rdparty/libprocess">Libprocess</a> message passing throughput</strong>: These improvements will be covered in a separate blog post.</li>
</ul>
<p>Before we dive into the master failover improvements, I would like to recognize and thank the following contributors:</p>
<ul>
<li><strong>Dmitry Zhuk</strong>: for writing <em>a lot</em> of patches for improving the master failover performance.</li>
<li><strong>Michael Park</strong>: for reviewing and shipping many of Dmitry’s more challenging patches.</li>
<li><strong>Yan Xu</strong>: for writing the master failover benchmark that was the basis for measuring the improvements.</li>
</ul>
<h2>Master Failover Time-To-Completion</h2>
<p>Our first area of focus was to improve the time it takes for a master failover to complete, where completion is defined as all of the agents successfully reregistering. Mesos is architected to use a centralized master with standby masters that participate in a quorum for high availability. For scalability reasons, the leading master stores the state of the cluster in-memory. During a master failover, the leading master needs to therefore re-build the in-memory state from all of the agents that reregister. During this time, the master is available to process other requests, but will be exposing only partial state to API consumers.</p>
<p>The rebuilding of the master’s in-memory state can be expensive for larger clusters, and so the focus of this effort was to improve the efficiency of this. Improvements were made via several areas, and only the highest-impact changes are listed below:</p>
<h3>Protobuf 3.5.0 Move Support</h3>
<p>We upgraded to protobuf 3.5.0 in order to gain move support. When we profiled the master, we found that it spent a lot of time copying protobuf messages during agent re-registration. This support allowed us to eliminate copies of protobuf messages while retaining value semantics.</p>
<h3>Move Support and Copy Elimination in Libprocess <code>dispatch</code> / <code>defer</code> / <code>install</code></h3>
<p>Libprocess provides several primitives for message passing:</p>
<ul>
<li><code>dispatch</code>: Provides the ability to post a messages to a local <code>Process</code></li>
<li><code>defer</code>: Provides a deferred <code>dispatch</code>. i.e. a function object that when invoked will issue a <code>dispatch</code>.</li>
<li><code>install</code>: Installs a handler for receiving a protobuf message.</li>
</ul>
<p>These primitives did not have move support, as they were originally added prior to the addition of C++11 support to the code-base. In order to eliminate copies, we enhanced these primitives to support moving arguments in and out.</p>
<p>This required introducing a new C++ utility, because <code>defer</code> takes on the same API as <code>std::bind</code> (e.g., placeholders). Specifically, the function object returned by <code>std::bind</code> does not move the bound arguments into the stored callable. In order to enable this, <code>defer</code> now uses a utility we introduced called <code>lambda::partial</code> rather than <code>std::bind</code>. <code>lambda::partial</code> performs partial function application similar to <code>std::bind</code> except the returned function object moves the bound arguments into the stored callable if the invocation is performed on an r-value function object.</p>
<h3>Copy Elimination in the Master</h3>
<p>With these previous enhancements in place, we were able to eliminate many of the expensive copies of protobuf messages performed by the master.</p>
<h3>Benchmark and Results</h3>
<p>We wrote a synthetic benchmark to simulate a master failover. This benchmark prepares all the messages that would be sent to the master by the agents that need to reregister:</p>
<ul>
<li>The benchmark uses synthetic agents in that they are just an actor that knows how to reregister with the master.</li>
<li>Each “agent” will send a configurable number of active and completed tasks belonging to a configurable number of active and completed frameworks.</li>
<li>Each task has 10 small labels to introduce metadata overhead.</li>
</ul>
<p>The benchmark has a few caveats:</p>
<ul>
<li>It does not use executors (this would show improved results over what is shown below, but for simplicity the benchmark omits them)</li>
<li>It uses local message passing, whereas a real cluster would be passing messages over HTTP.</li>
<li>It uses a quorum size of 1, so writes to the master’s registry occur only on single local log replica.</li>
<li>The synthetic agents do not retry their re-registration, whereas typically agents will retry with a backoff.</li>
</ul>
<p>This was tested on a 2015 Macbook Pro with 2.8 GHz Intel Core i7 processor. Mesos was configured using: <code>Apple LLVM version 9.0.0 (clang-900.0.38)</code>, with <code>-O2</code> enabled in 1.5.0.</p>
<p>The first results represent a cluster with 10 active tasks per agent across 5 frameworks, with no completed tasks. The results from 1,000 - 40,000 agents with 10,000 - 400,000 active tasks:</p>
<p><img src="/assets/img/documentation/1.3-1.5_master_failover_no_history.png" alt="1.3 - 1.5 Master Failover without Task History Graph" /></p>
<p>There was a reduction in the time-to-completion of ~80% due to a 450-500% improvement in throughput across 1.3.0 to 1.5.0.</p>
<p>The second results add task history: each agent also now contains 100 completed tasks across 5 completed frameworks. The results from 1,000 - 40,000 agents with 10,000 - 400,000 active tasks and 100,000 - 4,000,000 completed tasks are shown below:</p>
<p><img src="/assets/img/documentation/1.3-1.5_master_failover_with_history.png" alt="2.3 - 1.5 Master Failover with Task History Graph" /></p>
<p>This represents a reduction in time-to-completion of ~85% due to a 550-700% improvement in throughput across 1.3.0 to 1.5.0.</p>
<h2>Performance Working Group Roadmap</h2>
<p>We’re currently targeting the following areas for improvements:</p>
<ul>
<li><strong>Performance of the v1 API</strong>: Currently the v1 API can be significantly slower than the v0 API. We would like to reach parity, and ideally surpass the performance of the v0 API.
<ul>
<li><strong><a href="https://github.com/apache/mesos/tree/master/3rdparty/libprocess">Libprocess</a> HTTP performance</strong>: This will be undertaken as part of improving the v1 API performance, since it is HTTP-based.</li>
</ul>
</li>
<li><strong>Master state API performance</strong>: Currently, API queries of the master’s state are serviced by the same master actor that processes all of the messages from schedulers and agents. Since the query processing can block the master from processing other events, users need to be careful not to query the master excessively. In practice, the master gets queried quite heavily due to the presence of several tools that rely on the master’s state (e.g. DNS tooling, UIs, CLIs, etc) and so this is a critical problem for users. This effort will leverage the state streaming API to stream the state to a different actor that can serve the state API requests. This will ensure that expensive state queries do not affect the master’s ability to process events.</li>
</ul>
<p>If you are a user and would like to suggest some areas for performance improvement, please let us know by emailing <code>dev@apache.mesos.org</code>.</p>
http://mesos.apache.org/blog/mesos-1-2-3-released/
Apache Mesos 1.2.3 Released
2017-12-01T00:00:00+00:00Adam B
<p>The latest (and final) Mesos 1.2.x release, 1.2.3, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes important bug fixes on top of 1.2.2. It is recommended to use this version if you are considering using Mesos 1.2. More specifically, this release includes the following bug fixes:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6743">MESOS-6743</a> - Docker executor hangs forever if <code>docker stop</code> fails.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6950">MESOS-6950</a> - Launching two tasks with the same Docker image simultaneously may cause a staging dir never cleaned up.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7365">MESOS-7365</a> - Compile error with recent glibc.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7378">MESOS-7378</a> - Build failure with glibc 2.12.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7627">MESOS-7627</a> - Mesos slave stucks.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7652">MESOS-7652</a> - Docker image with universal containerizer does not work if WORKDIR is missing in the rootfs.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7744">MESOS-7744</a> - Mesos Agent Sends TASK_KILL status update to Master, and still launches task.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7783">MESOS-7783</a> - Framework might not receive status update when a just launched task is killed immediately.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7858">MESOS-7858</a> - Launching a nested container with namespace/pid isolation, with glibc < 2.25, may deadlock the LinuxLauncher and MesosContainerizer.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7863">MESOS-7863</a> - Agent may drop pending kill task status updates.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7865">MESOS-7865</a> - Agent may process a kill task and still launch the task.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7872">MESOS-7872</a> - Scheduler hang when registration fails.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7909">MESOS-7909</a> - Ordering dependency between ‘linux/capabilities’ and ‘docker/runtime’ isolator.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7926">MESOS-7926</a> - Abnormal termination of default executor can cause MesosContainerizer::destroy to fail.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7934">MESOS-7934</a> - OOM due to LibeventSSLSocket send incorrectly returning 0 after shutdown.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7968">MESOS-7968</a> - Handle <code>/proc/self/ns/pid_for_children</code> when parsing available namespace.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7969">MESOS-7969</a> - Handle cgroups v2 hierarchy when parsing /proc/self/cgroups.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7975">MESOS-7975</a> - The command/default/docker executor can incorrectly send a TASK_FINISHED update even when the task is killed.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7980">MESOS-7980</a> - Stout fails to compile with libc >= 2.26.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8051">MESOS-8051</a> - Killing TASK_GROUP fail to kill some tasks.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8080">MESOS-8080</a> - The default executor does not propagate missing task exit status correctly.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8135">MESOS-8135</a> - Masters can lose track of tasks' executor IDs.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.2.3">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.2.2 cluster to Mesos 1.2.3 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.2.3 from 1.1.x or 1.0.x.</p>
<p>NOTE: Since Mesos 1.2.1, the master does not allow 0.x agents to register.</p>
<h3>Try it out</h3>
<p>Please try out this release and let us know what you think. If you run into any issues, let us know on the <a href="https://mesos.apache.org/community">user mailing list and/or Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 12 contributors who made 1.2.3 possible:</p>
<p>Adam B, Alexander Rukletsov, Andrei Budnik, Benjamin Hindman, Benjamin Mahler, Gaston Kleiman, Gilbert Song, James Peach, Jie Yu, Kapil Arya, Neil Conway, and Qian Zhang.</p>
http://mesos.apache.org/blog/mesos-1-4-1-released/
Apache Mesos 1.4.1 Released
2017-11-16T00:00:00+00:00Kapil Arya
<p>The latest Mesos 1.4.x release, 1.4.1, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes important bug fixes and improvements on top of 1.4.0. It is recommended to use this version if you are considering using Mesos 1.4. More specifically, this release includes the following:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7873">MESOS-7873</a> - Expose <code>ExecutorInfo.ContainerInfo.NetworkInfo</code> in Mesos <code>state</code> endpoint.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7921">MESOS-7921</a> - ProcessManager::resume sometimes crashes accessing EventQueue.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7964">MESOS-7964</a> - Heavy-duty GC makes the agent unresponsive.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7968">MESOS-7968</a> - Handle <code>/proc/self/ns/pid_for_children</code> when parsing available namespace.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7969">MESOS-7969</a> - Handle cgroups v2 hierarchy when parsing /proc/self/cgroups.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7980">MESOS-7980</a> - Stout fails to compile with libc >= 2.26.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8051">MESOS-8051</a> - Killing TASK_GROUP fail to kill some tasks.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8080">MESOS-8080</a> - The default executor does not propagate missing task exit status correctly.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8090">MESOS-8090</a> - Mesos 1.4.0 crashes with 1.3.x agent with oversubscription</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8135">MESOS-8135</a> - Masters can lose track of tasks' executor IDs.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-8169">MESOS-8169</a> - Incorrect master validation forces executor IDs to be globally unique.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.4.1">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.4.0 cluster to Mesos 1.4.1 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.4.1 from 1.0.x, 1.2.x, or 1.3.x.</p>
<h3>Try it out</h3>
<p>Please try out this release and let us know what you think. If you run into any issues, let us know on the <a href="https://mesos.apache.org/community">user mailing list and/or Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 11 contributors who made 1.4.1 possible:</p>
<p>Benjamin Mahler, Chun-Hung Hsiao, Deepak Goel, Gaston Kleiman, James DeFelice, James Peach, Jiang Yan Xu, Kapil Arya, Michael Park, Qian Zhang, Zhitao Li</p>
http://mesos.apache.org/blog/mesos-1-4-0-released/
Apache Mesos 1.4.0 Released
2017-09-18T00:00:00+00:00Kapil Arya
<p>The latest Mesos release, 1.4.0, is now available for <a href="/downloads">download</a>. This release includes the following features and improvements:</p>
<ul>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-5116">MESOS-5116</a> -
The <code>disk/xfs</code> isolator now supports the <code>--enforce_container_disk_quota</code> flag
to efficiently measure disk usage without enforcing usage constraints. The
previous implementation always enforced the quota even if the operator might
not have wanted it.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-6223">MESOS-6223</a> -
With partition awareness, the agents are now allowed to reregister after they
have been marked Unreachable. Thus, they can recover their agent ID after a
host reboot. See docs/upgrades.md for details.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-6375">MESOS-6375</a> -
<em>Experimental</em> Support for hierarchical resource
allocation roles. Hierarchical roles allows delegation of resource
allocation policies (i.e. fair sharing and quota) further down the
hierarchy. For example, the “engineering” organization gets a 75%
share of the resources, but it’s up to the operators within the
“engineering” organization to figure out how to fairly share between
the “engineering/backend” team and the “engineering/frontend” team.
The same delegation applies for quota. NOTE: There are known issues
related to hierarchical roles (e.g. hierarchical quota allocation
is not implemented and quota will be over-allocated if used with
hierarchical roles, see: MESOS-7402) and thus it is not recommended
for production usage at this time.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-7418">MESOS-7418</a>,
<a href="https://issues.apache.org/jira/browse/MESOS-7088">MESOS-7088</a> -
File-based secrets are now supported for Mesos and Universal containerizer.
A new SecretResolver module kind is introduced to fetch and resolve any
secrets before they are made available to the tasks. Image-pull secrets are
also supported for Docker registry credentials.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-7477">MESOS-7477</a> -
Linux ambient capabilites are now supported, so frameworks can run tasks that
use ambient capabilites to grant limited additional privileged to tasks
without the requirement for matching file-based capabilities.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-7476">MESOS-7476</a>,
<a href="https://issues.apache.org/jira/browse/MESOS-7671">MESOS-7671</a> -
Support for frameworks and operators specifying Linux bounding capabilities in
order to limit the maximum privileges that a task may acquire.</p></li>
</ul>
<p>More than 200 other bug fixes and improvements made it into this release. For full release notes with all features and bug fixes, please refer to the <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.4.0">CHANGELOG</a>.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.3.0 cluster to Mesos 1.4.0 are straightforward. Please refer to the <a href="/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.4.0.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think.
If you run into any issues, please let us know on the <a href="/community">user mailing list, IRC, or Slack</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 53 contributors who made Mesos 1.4.0 possible:</p>
<p>Aaron Wood, Adam B, Alastair Montgomery, Alexander Rojas, Alexander Rukletsov, Anand Mazumdar, Andrei Budnik, Andrew Schwartzmeyer, Anindya Sinha, Armand Grillet, Avinash sridharan, Benjamin Bannier, Benjamin Hindman, Benjamin Mahler, Benno Evers, Chun-Hung Hsiao, Conrad Kleinespel, Dmitry Zhuk, Eric Chung, Gastón Kleiman, Gilbert Song, Greg Mann, haosdent huang, Haosdent Huang, hardy.jung, huangwenpeng, Jacob Janco, James Peach, Jan Schlicht, Jay Guo, Jeff Coffler, Jiang Yan Xu, Jie Yu, John Kordich, Joseph Wu, Kapil Arya, Kevin Klues, Li Li, Mao Geng, Megha Sharma, Michael Park, Neil Conway, Olivier Sallou, Qian Zhang, Quinn Leng, Shingo Kitayama, Silas Snider, Till Toenshoff, Tim Hansen, TobiLG, Vinod Kone, Zhitao Li, Zhongbo Tian</p>
http://mesos.apache.org/blog/mesos-1-1-3-released/
Apache Mesos 1.1.3 Released
2017-08-31T00:00:00+00:00AlexR
<p>The latest Mesos release, 1.1.3, is now available for
<a href="http://mesos.apache.org/downloads">download</a>. This release includes some
important bug fixes and improvements on top of 1.1.2. It is highly recommended
that users use this version if you are considering using Mesos 1.1. More
specifically, this release includes the following major fixes and improvements:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5187">MESOS-5187</a> - The filesystem/linux isolator does not set the permissions of the host_path.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6743">MESOS-6743</a> - Docker executor hangs forever if <code>docker stop</code> fails.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6950">MESOS-6950</a> - Launching two tasks with the same Docker image simultaneously may cause a staging dir never cleaned up.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7540">MESOS-7540</a> - Add an agent flag for executor re-registration timeout.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7569">MESOS-7569</a> - Allow “old” executors with half-open connections to be preserved during agent upgrade / restart.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7689">MESOS-7689</a> - Libprocess can crash on malformed request paths for libprocess messages.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7690">MESOS-7690</a> - The agent can crash when an unknown executor tries to register.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7581">MESOS-7581</a> - Fix interference of external Boost installations when using some unbundled dependencies.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7703">MESOS-7703</a> - Mesos fails to exec a custom executor when no shell is used.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7728">MESOS-7728</a> - Java HTTP adapter crashes JVM when leading master disconnects.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7770">MESOS-7770</a> - Persistent volume might not be mounted if there is a sandbox volume whose source is the same as the target of the persistent volume.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7777">MESOS-7777</a> - Agent failed to recover due to mount namespace leakage in Docker 1.12/1.13.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7796">MESOS-7796</a> - LIBPROCESS_IP isn’t passed on to the fetcher.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7830">MESOS-7830</a> - Sandbox_path volume does not have ownership set correctly.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7863">MESOS-7863</a> - Agent may drop pending kill task status updates.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7865">MESOS-7865</a> - Agent may process a kill task and still launch the task.</li>
</ul>
<p>Full release notes are available in the release
<a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.1.3">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.1.2 cluster to Mesos 1.1.3 are straightforward.
Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a>
for detailed information on upgrading to Mesos 1.1.3.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think. If you
run into any issues, please let us know on the
<a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 11 contributors who made 1.1.3 possible:</p>
<p>Aaron Wood, Alexander Rukletsov, Andrei Budnik, Benjamin Bannier, Benjamin Mahler, Chun-Hung Hsiao, Gastón Kleiman, Gilbert Song, Greg Mann, Jie Yu, and Qian Zhang.</p>
http://mesos.apache.org/blog/mesos-1-3-1-released/
Apache Mesos 1.3.1 Released
2017-08-15T00:00:00+00:00Michael Park
<p>The latest Mesos 1.3.x release, 1.3.1, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes important bug fixes and improvements on top of 1.3.0. It is recommended to use this version if you are considering using Mesos 1.3. More specifically, this release includes the following:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5187">MESOS-5187</a> - The filesystem/linux isolator does not set the permissions of the host_path.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7252">MESOS-7252</a> - Need to fix resource check in long-lived framework.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7429">MESOS-7429</a> - Allow isolators to inject task-specific environment variables.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7540">MESOS-7540</a> - Add an agent flag for executor re-registration timeout.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7546">MESOS-7546</a> - <code>WAIT_NESTED_CONTAINER</code> sometimes returns 404.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7569">MESOS-7569</a> - Allow “old” executors with half-open connections to be preserved during agent upgrade / restart.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7581">MESOS-7581</a> - Fix interference of external Boost installations when using some unbundled dependencies.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7689">MESOS-7689</a> - Libprocess can crash on malformed request paths for libprocess messages.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7690">MESOS-7690</a> - The agent can crash when an unknown executor tries to register.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7692">MESOS-7692</a> - Default environment variables defined in Docker image are not available in Mesos containerizer.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7703">MESOS-7703</a> - Mesos fails to exec a custom executor when no shell is used.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7728">MESOS-7728</a> - Java HTTP adapter crashes JVM when leading master disconnects.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7770">MESOS-7770</a> - Persistent volume might not be mounted if there is a sandbox volume whose source is the same as the target of the persistent volume.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7777">MESOS-7777</a> - Agent failed to recover due to mount namespace leakage in Docker 1.12/1.13.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7796">MESOS-7796</a> - <code>LIBPROCESS_IP</code> isn’t passed on to the fetcher.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7830">MESOS-7830</a> - Sandbox_path volume does not have ownership set correctly.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.3.1">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.3.0 cluster to Mesos 1.3.1 are straightforward.
Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade
guide</a> for detailed information on upgrading to Mesos 1.3.1 from 1.2.x, 1.1.x or 1.0.x.</p>
<h3>Try it out</h3>
<p>Please try out this release and let us know what you think. If you run into any issues, let us know on the <a href="https://mesos.apache.org/community">user mailing list and/or Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 52 contributors who made 1.3.1 possible:</p>
<p>Aaron Wood, Adam B, Alexander Rojas, Alexander Rukletsov, Anand Mazumdar, Andrei Budnik, Andrew Schwartzmeyer, Anindya Sinha, Armand Grillet, Avinash sridharan, Benjamin Bannier, Benjamin Hindman, Benjamin Mahler, Benno Evers, Chun-Hung Hsiao, Conrad Kleinespel, Dmitry Zhuk, Eric Chung, Gastón Kleiman, Gilbert Song, Greg Mann, Haosdent Huang, Jacob Janco, James Peach, Jan Schlicht, Jay Guo, Jeff Coffler, Jiang Yan Xu, Jie Yu, John Kordich, Joseph Wu, Kapil Arya, Kevin Klues, Li Li, Mao Geng, Megha Sharma, Michael Park, Neil Conway, Olivier Sallou, Qian Zhang, Quinn Leng, Shingo Kitayama, Silas Snider, Till Toenshoff, Tim Hansen, TobiLG, Vinod Kone, Zhitao Li, Zhongbo Tian, haosdent huang, hardy.jung, huangwenpeng</p>
http://mesos.apache.org/blog/mesos-1-2-2-released/
Apache Mesos 1.2.2 Released
2017-08-11T00:00:00+00:00Alexander Rojas
<p>The latest Mesos 1.2.x release, 1.2.2, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes important bug fixes and improvements on top of 1.2.1. It is recommended to use this version if you are considering using Mesos 1.2. More specifically, this release includes the following:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5187">MESOS-5187</a> - The filesystem/linux isolator does not set the permissions of the host_path.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7252">MESOS-7252</a> - Need to fix resource check in long-lived framework.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7546">MESOS-7546</a> - <code>WAIT_NESTED_CONTAINER</code> sometimes returns 404.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7569">MESOS-7569</a> - Allow “old” executors with half-open connections to be preserved during agent upgrade / restart.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7581">MESOS-7581</a> - Fix interference of external Boost installations when using some unbundled dependencies.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7689">MESOS-7689</a> - Libprocess can crash on malformed request paths for libprocess messages.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7690">MESOS-7690</a> - The agent can crash when an unknown executor tries to register.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7703">MESOS-7703</a> - Mesos fails to exec a custom executor when no shell is used.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7728">MESOS-7728</a> - Java HTTP adapter crashes JVM when leading master disconnects.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7770">MESOS-7770</a> - Persistent volume might not be mounted if there is a sandbox volume whose source is the same as the target of the persistent volume.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7777">MESOS-7777</a> - Agent failed to recover due to mount namespace leakage in Docker 1.12/1.13.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7796">MESOS-7796</a> - <code>LIBPROCESS_IP</code> isn’t passed on to the fetcher.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7830">MESOS-7830</a> - Sandbox_path volume does not have ownership set correctly.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7540">MESOS-7540</a> - Add an agent flag for executor re-registration timeout.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.2.2">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.2.1 cluster to Mesos 1.2.2 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.2.2 from 1.1.x or 1.0.x.</p>
<p>NOTE: Since Mesos 1.2.1, the master does not allow 0.x agents to register.</p>
<h3>Try it out</h3>
<p>Please try out this release and let us know what you think. If you run into any issues, let us know on the <a href="https://mesos.apache.org/community">user mailing list and/or Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 11 contributors who made 1.2.2 possible:</p>
<p>Aaron Wood, Adam B, Alexander Rukletsov, Benjamin Bannier, Benjamin Mahler, Chun-Hung Hsiao, Gastón Kleiman, Gilbert Song, Greg Mann, Jie Yu and Michael Park.</p>
http://mesos.apache.org/blog/mesos-1-2-1-released/
Apache Mesos 1.2.1 Released
2017-06-12T00:00:00+00:00Adam
<p>The latest Mesos 1.2.x release, 1.2.1, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes important bug fixes and improvements on top of 1.2.0. It is recommended to use this version if you are considering using Mesos 1.2. More specifically, this release includes the following:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1987">MESOS-1987</a> - Add support for SemVer build and prerelease labels to stout.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5028">MESOS-5028</a> - Copy provisioner cannot replace directory with symlink.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5172">MESOS-5172</a> - Registry puller cannot fetch blobs correctly from http Redirect 3xx urls.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6327">MESOS-6327</a> - Large docker images causes container launch failures: Too many levels of symbolic links.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6951">MESOS-6951</a> - Docker containerizer: mangled environment when env value contains LF byte.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6976">MESOS-6976</a> - Disallow (re-)registration attempts by old agents.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7133">MESOS-7133</a> - mesos-fetcher fails with openssl-related output.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7197">MESOS-7197</a> - Requesting tiny amount of CPU crashes master.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7208">MESOS-7208</a> - Persistent volume ownership is set to root when task is running with non-root user.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7210">MESOS-7210</a> - HTTP health check doesn’t work when mesos runs with –docker_mesos_image.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7232">MESOS-7232</a> - Add support to auto-load /dev/nvidia-uvm in the GPU isolator.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7237">MESOS-7237</a> - Enabling cgroups_limit_swap can lead to “invalid argument” error.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7261">MESOS-7261</a> - maintenance.html is missing during packaging.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7263">MESOS-7263</a> - User supplied task environment variables cause warnings in sandbox stdout.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7264">MESOS-7264</a> - Possibly duplicate environment variables should not leak values to the sandbox.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7265">MESOS-7265</a> - Containerizer startup may cause sensitive data to leak into sandbox logs.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7272">MESOS-7272</a> - Unified containerizer does not support docker registry version < 2.3.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7280">MESOS-7280</a> - Unified containerizer provisions docker image error with COPY backend.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7316">MESOS-7316</a> - Upgrading Mesos to 1.2.0 results in some information missing from the <code>/flags</code> endpoint.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7346">MESOS-7346</a> - Agent crashes if the task name is too long.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7350">MESOS-7350</a> - Failed to pull image from Nexus Registry due to signature missing.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7366">MESOS-7366</a> - Agent sandbox gc could accidentally delete the entire persistent volume content.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7368">MESOS-7368</a> - Documentation of framework role(s) in proto definition is confusing.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7383">MESOS-7383</a> - Docker executor logs possibly sensitive parameters.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7389">MESOS-7389</a> - Mesos 1.2.0 crashes with pre-1.0 Mesos agents.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7400">MESOS-7400</a> - The mesos master crashes due to an incorrect invariant check in the decoder.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7427">MESOS-7427</a> - Registry puller cannot fetch manifests from Amazon ECR: 405 Unsupported.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7429">MESOS-7429</a> - Allow isolators to inject task-specific environment variables.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7453">MESOS-7453</a> - glyphicons-halflings-regular.woff2 is missing in WebUI.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7464">MESOS-7464</a> - Recent Docker versions cannot be parsed by stout.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7471">MESOS-7471</a> - Provisioner recover should not always assume ‘rootfses’ dir exists.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7478">MESOS-7478</a> - Pre-1.2.x master does not work with 1.2.x agent.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7484">MESOS-7484</a> - VersionTest.ParseInvalid aborts on Windows.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.2.1">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.2.0 cluster to Mesos 1.2.1 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.2.1 from 1.1.x or 1.0.x.</p>
<p>NOTE: In Mesos 1.2.1, the master will no longer allow 0.x agents to register. Interoperability between 1.1+ masters and 0.x agents has never been supported;
however, it was not explicitly disallowed, either. Starting with this release of Mesos, registration attempts by 0.x Mesos agents will be ignored.</p>
<h3>Try it out</h3>
<p>Please try out this release and let us know what you think. If you run into any issues, let us know on the <a href="https://mesos.apache.org/community">user mailing list and/or Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 16 contributors who made 1.2.1 possible:</p>
<p>Aaron Wood, Adam B, Alexander Rukletsov, Anand Mazumdar, Andrew Schwartzmeyer, Anindya Sinha, Benjamin Bannier, Benjamin Mahler, Chun-Hung Hsiao, Deshi Xiao, Gilbert Song, Haosdent Huang, Jie Yu, Kevin Klues, Neil Conway, and Till Toenshoff.</p>
http://mesos.apache.org/blog/mesos-1-3-0-released/
Apache Mesos 1.3.0 Released
2017-06-07T00:00:00+00:00Neil Conway
<p>The latest Mesos release, 1.3.0, is now available for <a href="/downloads">download</a>. This release includes the following features and improvements:</p>
<ul>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-1763">MESOS-1763</a> -
Support for frameworks to receive resources for multiple roles. This allows
“multi-user” frameworks to leverage the role-based resource allocation in
mesos. Prior to this support, one had to run multiple instances of a
single-user framework to achieve multi-user resource allocation, or
implement multi-user resource allocation in the framework.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-6365">MESOS-6365</a> -
Authentication and authorization support for HTTP executors. A new
<code>--authenticate_http_executors</code> agent flag enables required authentication
on the HTTP executor API. A new <code>--executor_secret_key</code> flag sets a key file
to be used when generating and authenticating default tokens that are passed
to HTTP executors. Note that enabling these flags after upgrade is
disruptive to HTTP executors that were launched before the upgrade; see
‘docs/authentication.md’ for more information on these flags and the
recommended upgrade procedure. Implicit authorization rules have been added
which allow an authenticated executor to make executor API calls as that
executor and make operator API calls which affect that executor’s
container. See ‘docs/authorization.md’ for more information on these
implicit authorization rules.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-6627">MESOS-6627</a> - Support for
frameworks to modify the role(s) they are subscribed to. This is essential
to supporting “multi-user” frameworks (see
<a href="https://issues.apache.org/jira/browse/MESOS-1763">MESOS-1763</a>) in that
roles are expected to come and go over time (e.g. new employees join, new
teams are formed, employees leave, teams are disbanded, etc).</p></li>
</ul>
<p>More than 140 other bug fixes and improvements made it into this release. For full release notes with all features and bug fixes, please refer to the <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.3.0">CHANGELOG</a>.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.2.0 cluster to Mesos 1.3.0 are straightforward. Please refer to the <a href="/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.3.0.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think.
If you run into any issues, please let us know on the <a href="/community">user mailing list, IRC, or Slack</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 55 contributors who made Mesos 1.3.0 possible:</p>
<p>Aaron Wood, Abhishek Dasgupta, Adam B, Alex Clemmer, Alexander Rojas, Alexander Rukletsov, Anand Mazumdar, Andrew Schwartzmeyer, Andy Pang, Anindya Sinha, Anthony Sottile, Armand Grillet, Avinash sridharan, Ayanampudi Varsha, Benjamin Bannier, Benjamin Mahler, Benno Evers, Budnik Andrei, Chris Cao, Chun-Hung Hsiao, Daniel Pravat, Deshi Xiao, Dmitry Zhuk, Eric Chung, Gastón Kleiman, Gilbert Song, Greg Mann, Guangya Liu, Haosdent Huang, Haris Choudhary, Ilya Pronin, James Peach, Jan Schlicht, Jay Guo, Jeff Coffler, Jiang Yan Xu, Jie Yu, John Kordich, Joseph Wu, Kamil Wargula, Kapil Arya, Kevin Klues, Klaus Ma, Michael Park, Neil Conway, Pierre Cheynier, Qian Zhang, Srinivas Brahmaroutu, Till Toenshoff, Tomasz Janiszewski, Vinod Kone, Yan Xu, Zhitao Li, Zhongbo Tian, and ondrej.smola.</p>
http://mesos.apache.org/blog/mesos-1-0-4-released/
Apache Mesos 1.0.4 Released
2017-05-03T00:00:00+00:00Vinod Kone
<p>The latest Mesos release, 1.0.4, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes some important bug fixes and improvements on top of 1.0.0. It is highly recommended to use this version if you are considering using Mesos 1.0. More specifically, this release includes the following fixes and improvements:</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-2537">MESOS-2537</a> - AC_ARG_ENABLED checks are broken</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6606">MESOS-6606</a> - Reject optimized builds with libcxx before 3.9</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-7008">MESOS-7008</a> - Quota not recovered from registry in empty cluster.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-7265">MESOS-7265</a> - Containerizer startup may cause sensitive data to leak into sandbox logs.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-7366">MESOS-7366</a> - Agent sandbox gc could accidentally delete the entire persistent volume content.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-7383">MESOS-7383</a> - Docker executor logs possibly sensitive parameters.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-7422">MESOS-7422</a> - Docker containerizer should not leak possibly sensitive data to agent log.</p>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.0.4">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.0.0 cluster to Mesos 1.0.4 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.0.4.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think. If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing ld Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 8 contributors who made 1.0.4 possible</p>
<p>Alexander Rukletsov, Benjamin Bannier, James Peach, Jie Yu, Kapil Arya, Neil Conway, Till Toenshoff, Vinod Kone.</p>
http://mesos.apache.org/blog/mesos-1-1-1-released/
Apache Mesos 1.1.1 Released
2017-03-14T00:00:00+00:00AlexR
<p>The latest Mesos release, 1.1.1, is now available for
<a href="http://mesos.apache.org/downloads">download</a>. This release includes some
important bug fixes and improvements on top of 1.1.0. It is highly recommended
that users use this version if you are considering using Mesos 1.1. More
specifically, this release includes the following major fixes and improvements:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6002">MESOS-6002</a> - The whiteout file cannot be removed correctly using aufs backend.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6010">MESOS-6010</a> - Docker registry puller shows decode error “No response decoded”.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6142">MESOS-6142</a> - Frameworks may RESERVE for an arbitrary role.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6360">MESOS-6360</a> - The handling of whiteout files in provisioner is not correct.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6411">MESOS-6411</a> - Add documentation for CNI port-mapper plugin.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6526">MESOS-6526</a> - <code>mesos-containerizer launch --environment</code> exposes executor env vars in <code>ps</code>.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6571">MESOS-6571</a> - Add “–task” flag to mesos-execute</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6597">MESOS-6597</a> - Include v1 Operator API protos in generated JAR and python packages.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6606">MESOS-6606</a> - Reject optimized builds with libcxx before 3.9</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6621">MESOS-6621</a> - SSL downgrade path will CHECK-fail when using both temporary and persistent sockets</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6624">MESOS-6624</a> - Master WebUI does not work on Firefox 45</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6676">MESOS-6676</a> - Always re-link with scheduler during re-registration.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6848">MESOS-6848</a> - The default executor does not exit if a single task pod fails.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6852">MESOS-6852</a> - Nested container’s launch command is not set correctly in docker/runtime isolator.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6917">MESOS-6917</a> - Segfault when the executor sets an invalid UUID when sending a status update.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7008">MESOS-7008</a> - Quota not recovered from registry in empty cluster.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-7133">MESOS-7133</a> - mesos-fetcher fails with openssl-related output.</li>
</ul>
<p>Full release notes are available in the release
<a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.1.1">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.1.0 cluster to Mesos 1.1.1 are straightforward.
Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a>
for detailed information on upgrading to Mesos 1.1.1.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think. If you
run into any issues, please let us know on the
<a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 15 contributors who made 1.1.1 possible:</p>
<p>Aaron Wood, Alexander Rukletsov, Anand Mazumdar, Avinash sridharan, Benjamin Bannier, Gastón Kleiman, Gilbert Song, Haosdent Huang, Jan Schlicht, Jiang Yan Xu, Jie Yu, Joseph Wu, Michael Park, Neil Conway, Qian Zhang, Till Toenshoff, Vijay Srinivasaraghavan, Vinod Kone, haosdent huang.</p>
http://mesos.apache.org/blog/mesos-1-2-0-released/
Apache Mesos 1.2.0 Released
2017-03-08T00:00:00+00:00Adam
<p>The latest Mesos release, 1.2.0, is now available for <a href="/downloads">download</a>. This release includes the following features and improvements:</p>
<ul>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-5931">MESOS-5931</a> -
<strong>Experimental</strong> support for auto backend in Mesos Containerizer,
prefering overlayfs then aufs. Please note that the bind backend needs to be
specified explicitly through the agent flag <code>--image_provisioner_backend</code>
since it requires the sandbox already existed.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-6402">MESOS-6402</a> -
<strong>Experimental</strong> support for rlimit in Mesos containerizer.
The isolator adds support for setting POSIX resource limits (rlimits) for
containers launched using the Mesos containerizer. POSIX rlimits can be used
to control the resources a process can consume. See
<a href="/documentation/latest/isolators/posix-rlimits/">docs</a> for details.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-6419">MESOS-6419</a> -
<strong>Experimental</strong>: Teardown unregistered frameworks. The master
now treats recovered frameworks very similarly to frameworks that are registered
but currently disconnected. For example, recovered frameworks will be reported
via the normal “frameworks” key when querying HTTP endpoints. This means there
is no longer a concept of “orphan tasks”: if the master knows about a task, the
task will be running under a framework. Similarly, “teardown” operations on
recovered frameworks will now work correctly.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-6460">MESOS-6460</a> -
<strong>Experimental</strong>: Container Attach and Exec. This feature adds new
<a href="/documentation/latest/operator-http-api">Agent APIs</a> for attaching a remote
client to the stdin, stdout, and stderr of a running Mesos task, as well as
an API for launching new processes inside the same container as a running
Mesos task and attaching to its stdin, stdout, and stderr. At a high level,
these APIs mimic functionality similar to docker attach and docker exec.
The primary motivation for such functionality is to enable users to debug
their running Mesos tasks.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-6758">MESOS-6758</a> -
<strong>Experimental</strong> support for ‘Basic’ auth docker private registry
on Mesos Containerizer. Until now, the mesos containerizer always assumed
Bearer auth, but we now also support basic auth for private registries. Please
note that the AWS ECS uses Basic authorization but it does not work yet due to
the redirect issue <a href="https://issues.apache.org/jira/browse/MESOS-5172">MESOS-5172</a>.</p></li>
</ul>
<p>More than 200 other bug fixes and improvements made it into this release. For full release notes with all features and bug fixes, please refer to the <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.2.0">CHANGELOG</a>.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.1.0 cluster to Mesos 1.2.0 are straightforward. There are just some minor, backwards compatible deprecations.
Please refer to the <a href="/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.2.0.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think.
If you run into any issues, please let us know on the <a href="/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 56 contributors who made Mesos 1.2.0 possible:</p>
<p>Aaron Wood, Abhishek Dasgupta, Adam B, Alexander Rojas, Alexander Rukletsov, Alex Clemmer, Anand Mazumdar, Andrew Schwartzmeyer, Anindya Sinha, Armand Grillet, Avinash Sridharan, Benjamin Bannier, Benjamin Hindman, Benjamin Mahler, Bruce Merry, Chengwei Yang, Daniel Pravat, David Forsythe, Dmitry Zhuk, Gastón Kleiman, Gilbert Song, Greg Mann, Guangya Liu, Haosdent Huang, Ilya Pronin, Jacob Janco, James Peach, Jan Schlicht, Jay Guo, Jeff Malnick, Jiang Yan Xu, Jie Yu, Joerg Schad Johannes Unterstein, Joris Van Remoortere, Joseph Wu, Kevin Klues, Lior Zeno, Manuwela Kanade, Megha Sharma, Michael Park, Miguel Bernadin, Neil Conway, Nicholas Sun, Qian Zhang, Ronald Petty, Santhosh Kumar Shanmugham, Sivaram Kannan, Srinivas Brahmaroutu, Thomas Maurice, Till Toenshoff, Tomasz Janiszewski, Vijay Srinivasaraghavan, Vinod Kone, Yubo Li, and Zhitao Li.</p>
http://mesos.apache.org/blog/mesos-1-0-3-released/
Apache Mesos 1.0.3 Released
2017-02-06T00:00:00+00:00Vinod Kone
<p>The latest Mesos release, 1.0.3, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes some important bug fixes and improvements on top of 1.0.0. It is highly recommended to use this version if you are considering using Mesos 1.0. More specifically, this release includes the following fixes and improvements:</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6052">MESOS-6052</a> - Unable to launch containers on CNI networks on CoreOS</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6142">MESOS-6142</a> - Frameworks may RESERVE for an arbitrary role.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6621">MESOS-6621</a> - SSL downgrade path will CHECK-fail when using both temporary and persistent sockets</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6676">MESOS-6676</a> - Always re-link with scheduler during re-registration.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6917">MESOS-6917</a> - Segfault when the executor sets an invalid UUID when sending a status update.</p>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.0.3">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.0.0 cluster to Mesos 1.0.3 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.0.3.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think. If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing ld Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 9 contributors who made 1.0.3 possible</p>
<p>Aaron Wood, Alexander Rukletsov, Avinash sridharan, Benjamin Bannier, Gastón Kleiman, Joseph Wu, Michael Park,Neil Conway, Vinod Kone</p>
http://mesos.apache.org/blog/mesos-0-28-3-released/
Apache Mesos 0.28.3 Released
2016-12-20T00:00:00+00:00Anand Mazumdar
<p>The latest Mesos release, 0.28.3, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes some important bug fixes and improvements on top of 0.28.2. It is highly recommended that users use this version if you are considering using Mesos 0.28. More specifically, this release includes the following major fixes and improvements:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2043">MESOS-2043</a> - Framework auth fail with timeout error and never get authenticated</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5571">MESOS-5571</a> - Scheduler JNI throws exception when the major versions of JAR and libmesos don’t match.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5691">MESOS-5691</a> - SSL downgrade support will leak sockets in CLOSE_WAIT status.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5698">MESOS-5698</a> - Quota sorter not updated for resource changes at agent.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5723">MESOS-5723</a> - SSL-enabled libprocess will leak incoming links to forks.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5763">MESOS-5763</a> - Task stuck in fetching is not cleaned up after –executor_registration_timeout.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5913">MESOS-5913</a> - Stale socket FD usage when using libevent + SSL.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5943">MESOS-5943</a> - Incremental http parsing of URLs leads to decoder error.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5986">MESOS-5986</a> - SSL Socket CHECK can fail after socket receives EOF.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6104">MESOS-6104</a> - Potential FD double close in libevent’s</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6142">MESOS-6142</a> - Frameworks may RESERVE for an arbitrary role.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6233">MESOS-6233</a> - Master CHECK fails during recovery while relinking to other masters.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6299">MESOS-6299</a> - Master doesn’t remove task from pending when it is invalid.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6457">MESOS-6457</a> - Tasks shouldn’t transition from TASK_KILLING to TASK_RUNNING.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6502">MESOS-6502</a> - _version uses incorrect MESOS<em>{MAJOR,MINOR,PATCH}</em>VERSION in libmesos java binding.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6527">MESOS-6527</a> - Memory leak in the libprocess request decoder.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-6621">MESOS-6621</a> - SSL downgrade path will CHECK-fail when using both temporary and persistent sockets.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.28.3">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 0.28.2 cluster to Mesos 0.28.3 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 0.28.3.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think. If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 15 contributors who made 0.28.3 possible:</p>
<p>Anand Mazumdar, Benjamin Bannier, Benjamin Mahler, David Robinson, Gastón Kleiman, Gilbert Song, Greg Mann, Joris Van Remoortere, Joseph Wu, Neil Conway, Silas Snider, Vinod Kone, Yan Xu, Zhitao Li, Zhou Xing</p>
http://mesos.apache.org/blog/mesos-1-0-2-released/
Apache Mesos 1.0.2 Released
2016-11-15T00:00:00+00:00Vinod Kone
<p>The latest Mesos release, 1.0.2, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes some important bug fixes and improvements on top of 1.0.0. It is highly recommended to use this version if you are considering using Mesos 1.0. More specifically, this release includes the following fixes and improvements:</p>
<p>** Bug</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-4638">MESOS-4638</a> - Versioning preprocessor macros.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-4973">MESOS-4973</a> - Duplicates in ‘unregistered_frameworks’ in /state</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-4975">MESOS-4975</a> - mesos::internal::master::Slave::tasks can grow unboundedly.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5613">MESOS-5613</a> - mesos-local fails to start if MESOS_WORK_DIR isn’t set.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6013">MESOS-6013</a> - Use readdir instead of readdir_r.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6026">MESOS-6026</a> - Tasks mistakenly marked as FAILED due to race b/w sendExecutorTerminatedStatusUpdate() and _statusUpdate().</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6074">MESOS-6074</a> - Master check failure if the metrics endpoint is polled soon after it starts.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6100">MESOS-6100</a> - Make fails compiling 1.0.1.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6104">MESOS-6104</a> - Potential FD double close in libevent’s implementation of <code>sendfile</code>.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6118">MESOS-6118</a> - Agent would crash with docker container tasks due to host mount table read.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6122">MESOS-6122</a> - Mesos slave throws systemd errors even when passed a flag to disable systemd.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6152">MESOS-6152</a> - Resource leak in libevent_ssl_socket.cpp.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6212">MESOS-6212</a> - Validate the name format of mesos managed docker containers.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6216">MESOS-6216</a> - LibeventSSLSocketImpl::create is not safe to call concurrently with os::getenv.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6233">MESOS-6233</a> - Master CHECK fails during recovery while relinking to other masters.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6234">MESOS-6234</a> - Potential socket leak during Zookeeper network changes.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6245">MESOS-6245</a> - Driver based schedulers performing explicit acknowledgements cannot acknowledge updates from HTTP based executors.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6246">MESOS-6246</a> - Libprocess links will not generate an ExitedEvent if the socket creation fails.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6269">MESOS-6269</a> - CNI isolator doesn’t activate loopback interface.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6274">MESOS-6274</a> - Agent should not allow HTTP executors to re-subscribe before containerizer recovery is done.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6324">MESOS-6324</a> - CNI should not use <code>ifconfig</code> in executors <code>pre_exec_command</code>.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6391">MESOS-6391</a> - Command task’s sandbox should not be owned by root if it uses container image.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6393">MESOS-6393</a> - Deprecated SSL_ environment variables are non functional already.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6420">MESOS-6420</a> - Mesos Agent leaking sockets when port mapping network isolator is ON.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6446">MESOS-6446</a> - WebUI redirect doesn’t work with stats from /metric/snapshot.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6457">MESOS-6457</a> - Tasks shouldn’t transition from TASK_KILLING to TASK_RUNNING.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6461">MESOS-6461</a> - Duplicate framework ids in /master/frameworks endpoint ‘unregistered_frameworks’.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6502">MESOS-6502</a> - <em>version uses incorrect MESOS</em>{MAJOR,MINOR,PATCH}_VERSION in libmesos java binding.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6527">MESOS-6527</a> - Memory leak in the libprocess request decoder.</p>
<p>** Improvement</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6075">MESOS-6075</a> - Avoid libprocess functions in <code>mesos-containerizer launch</code>.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-6299">MESOS-6299</a> - Master doesn’t remove task from pending when it is invalid.</p>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.0.2">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.0.0 cluster to Mesos 1.0.2 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.0.2.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think. If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing ld Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 20 contributors who made 1.0.2 possible</p>
<p>Alexander Rukletsov, Ammar Askar, Anand Mazumdar, Avinash sridharan, Benjamin Bannier, Benjamin Hindman, Benjamin Mahler, Gastón Kleiman, Jiang Yan Xu, Jie Yu, Joris Van Remoortere, Joseph Wu, Kevin Klues, Manuwela Kanade, Neil Conway, Santhosh Kumar Shanmugham, Till Toenshoff, Vinod Kone, Zhitao Li, haosdent huang.</p>
http://mesos.apache.org/blog/mesos-1-1-0-released/
Apache Mesos 1.1.0 Released
2016-11-10T00:00:00+00:00Till Toenshoff
<p>The latest Mesos release, 1.1.0, is now available for <a href="http://mesos.apache.org/downloads">download</a>.</p>
<p>This release includes the following features and improvements:</p>
<ul>
<li><p>[MESOS-2449] - <strong>Experimental</strong> support for launching a group of tasks
via a new <code>LAUNCH_GROUP</code> Offer operation. Mesos will guarantee that either
all tasks or none of the tasks in the group are delivered to the executor.
Executors receive the task group via a new <code>LAUNCH_GROUP</code> event.</p></li>
<li><p>[MESOS-2533] - <strong>Experimental</strong> support for HTTP and HTTPS health checks.
Executors may now use the updated <code>HealthCheck</code> protobuf to implement
HTTP(S) health checks. Both default executors (command and docker) leverage
<code>curl</code> binary for sending HTTP(S) requests and connect to <code>127.0.0.1</code>,
hence a task must listen on all interfaces. On Linux, for BRIDGE and USER
modes, docker executor enters the task’s network namespace.</p></li>
<li><p>[MESOS-3421] - <strong>Experimental</strong> Support sharing of resources across
containers. Currently persistent volumes are the only resources allowed to
be shared.</p></li>
<li><p>[MESOS-3567] - <strong>Experimental</strong> support for TCP health checks. Executors
may now use the updated <code>HealthCheck</code> protobuf to implement TCP health
checks. Both default executors (command and docker) connect to <code>127.0.0.1</code>,
hence a task must listen on all interfaces. On Linux, for BRIDGE and USER
modes, docker executor enters the task’s network namespace.</p></li>
<li><p>[MESOS-4324] - Allow tasks to access persistent volumes in either a
read-only or read-write manner. Using a volume in read-only mode can
simplify sharing that volume between multiple tasks on the same agent.</p></li>
<li><p>[MESOS-5275] - <strong>Experimental</strong> support for linux capabilities. Frameworks
or operators now have fine-grained control over the capabilities that a
container may have. This allows a container to run as root, but not have all
the privileges associated with the root user (e.g., CAP_SYS_ADMIN).</p></li>
<li><p>[MESOS-5344] - <strong>Experimental</strong> support for partition-aware Mesos
frameworks. In previous Mesos releases, when an agent is partitioned from
the master and then reregisters with the cluster, all tasks running on the
agent are terminated and the agent is shutdown. In Mesos 1.1, partitioned
agents will no longer be shutdown when they reregister with the master. By
default, tasks running on such agents will still be killed (for backward
compatibility); however, frameworks can opt-in to the new PARTITION_AWARE
capability. If they do this, their tasks will not be killed when a partition
is healed. This allows frameworks to define their own policies for how to
handle partitioned tasks. Enabling the PARTITION_AWARE capability also
introduces a new set of task states: TASK_UNREACHABLE, TASK_DROPPED,
TASK_GONE, TASK_GONE_BY_OPERATOR, and TASK_UNKNOWN. These new states are
intended to eventually replace the TASK_LOST state.</p></li>
<li><p>[MESOS-5788] - <strong>Experimental</strong> support for Java scheduler adapter. This
adapter allows framework developers to toggle between the old/new API
(driver/scheduler library) implementations, thereby allowing them to easily
transition their frameworks to the new v1 Scheduler API.</p></li>
<li><p>[MESOS-6014] - <strong>Experimental</strong> A new port-mapper CNI plugin, the
<code>mesos-cni-port-mapper</code> has been introduced. For Mesos containers, with the
CNI port-mapper plugin, users can now expose container ports through host
ports using DNAT. This is especially useful when Mesos containers are
attached to isolated CNI networks such as private bridge networks, and the
services running in the container needs to be exposed outside these
isolated networks.</p></li>
<li><p>[MESOS-6077] - <strong>Experimental</strong> A new default executor is introduced which
frameworks can use to launch task groups as nested containers. All the
nested containers share resources likes cpu, memory, network and volumes.</p></li>
</ul>
<h4>Deprecations</h4>
<ul>
<li><p>The following metrics are deprecated and will be removed in Mesos 1.4:
master/slave_shutdowns_scheduled,
master/slave_shutdowns_canceled,
slave_shutdowns_completed.
As of Mesos 1.1.0, these metrics will always be zero. The following new
metrics have been introduced as replacements:
master/slave_unreachable_scheduled,
master/slave_unreachable_canceled,
master/slave_unreachable_completed.</p></li>
<li><p>[MESOS-5955] - Health check binary “mesos-health-check” is removed.</p></li>
<li><p>[MESOS-6371] - Remove the ‘recover()’ interface in ‘ContainerLogger’.</p></li>
</ul>
<h4>Additional API Changes</h4>
<ul>
<li><p>[MESOS-6204] - A new agent flag called <code>--runtime_dir</code>. Unlike
<code>--work_dir</code> which persists data across reboots, <code>--runtime_dir</code> is designed
to checkpoint state that should persist across agent restarts, but not
across reboots. By default this flag is set to <code>/var/run/mesos</code> when run as
root and <code>os::temp/mesos/runtime/</code> when run as non-root.</p></li>
<li><p>[MESOS-6220] - HTTP handler failures should result in 500 rather than
503 responses. This means that when using the master or agent endpoints,
failures will now result in a <code>500 Internal Server Error</code> rather than a
<code>503 Service Unavailable</code>.</p></li>
<li><p>[MESOS-6241] - New API calls (LAUNCH_NESTED_CONTAINER,
KILL_NESTED_CONTAINER and WAIT_NESTED_CONTAINER) have been added to the
v1 Agent API to manage nested containers within an executor container.</p></li>
</ul>
<h4>Unresolved Critical Issues</h4>
<ul>
<li><p>[MESOS-3794] - Master should not store arbitrarily sized data in ExecutorInfo.</p></li>
<li><p>[MESOS-4642] - Mesos Agent Json API can dump binary data from log files out as invalid JSON.</p></li>
<li><p>[MESOS-5396] - After failover, master does not remove agents with same UPID.</p></li>
<li><p>[MESOS-5856] - Logrotate ContainerLogger module does not rotate logs when run as root with <code>--switch_user</code>.</p></li>
<li><p>[MESOS-6142] - Frameworks may RESERVE for an arbitrary role.</p></li>
<li><p>[MESOS-6327] - Large docker images causes container launch failures: Too many levels of symbolic links.</p></li>
<li><p>[MESOS-6360] - The handling of whiteout files in provisioner is not correct.</p></li>
<li><p>[MESOS-6419] - The ‘master/teardown’ endpoint should support tearing down ‘unregistered_frameworks’.</p></li>
<li><p>[MESOS-6432] - Roles with quota assigned can “game” the system to receive excessive resources.</p></li>
</ul>
<h4>All Experimental Features</h4>
<ul>
<li><p>[MESOS-2449] - Support group of tasks (Pod) constructs and API in Mesos.</p></li>
<li><p>[MESOS-2533] - Support HTTP checks in Mesos.</p></li>
<li><p>[MESOS-3094] - Mesos on Windows.</p></li>
<li><p>[MESOS-3421] - Support sharing of resources across task instances.</p></li>
<li><p>[MESOS-3567] - Support TCP checks in Mesos.</p></li>
<li><p>[MESOS-4312] - Porting Mesos on Power (ppc64le).</p></li>
<li><p>[MESOS-4355] - Implement isolator for Docker volume.</p></li>
<li><p>[MESOS-4641] - Support Container Network Interface (CNI).</p></li>
<li><p>[MESOS-4791] - Operator API v1.</p></li>
<li><p>[MESOS-4828] - XFS disk quota isolator.</p></li>
<li><p>[MESOS-5275] - Add capabilities support for unified containerizer.</p></li>
<li><p>[MESOS-5344] - Partition-aware Mesos frameworks.</p></li>
<li><p>[MESOS-5788] - Added JAVA API adapter for seamless transition to new scheduler API.</p></li>
<li><p>[MESOS-6014] - Added port mapping CNI plugin.</p></li>
<li><p>[MESOS-6077] - Added a default (task group) executor.</p></li>
</ul>
<p>Furthermore, several bugfixes and improvements made it into this release.
For full release notes with all features and bug fixes, please refer to the <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.1.0">CHANGELOG</a>.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.0.0 cluster to Mesos 1.1.0 are straightforward. There are just some minor, backwards compatible deprecations.
Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.1.0.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think.
If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 73 contributors who made 1.1.0 possible:</p>
<p>Aaron Wood, Abhishek Dasgupta, Alberto, Aleksandar Prokopec, Alex Clemmer, Alexander Rojas, Alexander Rukletsov, Ammar Askar, Anand Mazumdar, Andy Pang, Anindya Sinha, Artem Harutyunyan, Attila Szarvas, Avinash sridharan, Bart Spaans, Benjamin Bannier, Benjamin Hindman, Benjamin Mahler, Charles Allen, ChrisPaprocki, Daniel Pravat, Dario Rexin, Dave Lester, Gaojin CAO, Gastón Kleiman, Gilbert Song, Giulio Eulisse, Graham Taylor, Greg Mann, Guangya Liu, Haris Choudhary, Ian Babrou, Ilya Pronin, Jacob Janco, Jan Schlicht, Jay Guo, Jiang Yan Xu, Jie Yu, Joerg Schad, Joris Van Remoortere, Joseph Wu, Ken Sipe, Kevin Klues, Klaus Ma, Kunal Thakar, Lawrence Wu, Lijun Tang, Lukas Loesche, Mao Geng, Megha Sharma, Michael Park, Miguel Bernadin, Neil Conway, Pierre Cheynier, Qian Zhang, Roger Ignazio, Santhosh Kumar Shanmugham, Srinivas Brahmaroutu, Till Toenshoff, Tomasz Janiszewski, Vinod Kone, Will Rouesnel, Yong Tang, Yongqiao Wang, Yubo Li, Zhitao Li, anthony caiafa, gnolan, haosdent huang, janisz, uzyexe, zhou xing</p>
http://mesos.apache.org/blog/mesos-1-0-1-released/
Apache Mesos 1.0.1 Released
2016-08-23T00:00:00+00:00Vinod Kone
<p>The latest Mesos release, 1.0.1, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes some important bug fixes and improvements on top of 1.0.0. It is highly recommended to use this version if you are considering using Mesos 1.0. More specifically, this release includes the following fixes and improvements:</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5388">MESOS-5388</a> - MesosContainerizerLaunch flags execute arbitrary commands via shell.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5862">MESOS-5862</a> - External links to .md files broken.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5911">MESOS-5911</a> - Webui redirection to leader in browser does not work</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5913">MESOS-5913</a> - Stale socket FD usage when using libevent + SSL.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5922">MESOS-5922</a> - mesos-agent –help exit status is 1</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5923">MESOS-5923</a> - Ubuntu 14.04 LTS GPU Isolator “/run” directory is noexec</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5927">MESOS-5927</a> - Unable to run “scratch” Dockerfiles with Unified Containerizer.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5928">MESOS-5928</a> - Agent’s ‘–version’ flag doesn’t work</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5930">MESOS-5930</a> - Orphan tasks can show up as running after they have finished.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5943">MESOS-5943</a> - Incremental http parsing of URLs leads to decoder error</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5945">MESOS-5945</a> - NvidiaVolume::create() should check for root before creating volume</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5959">MESOS-5959</a> - All non-root tests fail on GPU machine</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5969">MESOS-5969</a> - Linux ‘MountInfoTable’ entries not sorted as expected</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5982">MESOS-5982</a> - NvidiaVolume errors out if any binary is missing</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5986">MESOS-5986</a> - SSL Socket CHECK can fail after socket receives EOF</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5988">MESOS-5988</a> - PollSocketImpl can write to a stale fd.</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5830">MESOS-5830</a> - Make a sweep to trim excess space around angle brackets</p>
<p><a href="https://issues.apache.org/jira/browse/MESOS-5970">MESOS-5970</a> - Remove HTTP_PARSER_VERSION_MAJOR < 2 code in decoder.</p>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=1.0.1">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 1.0.0 cluster to Mesos 1.0.1 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 1.0.1.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think. If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing ld Slack/IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 12 contributors who made 1.0.1 possible</p>
<p>Anand Mazumdar, Benjamin Mahler, Gilbert Song, Greg Mann, Jiang Yan Xu, Jie Yu, Joris Van Remoortere, Kevin Klues, Pierre Cheynier, Vinod Kone, haosdent huang and zhou xing.</p>
http://mesos.apache.org/blog/mesos-1-0-0-released/
Apache Mesos 1.0.0 Released
2016-07-27T00:00:00+00:00Vinod Kone
<p>Mesos 1.0 release is finally here for <a href="http://mesos.apache.org/downloads">download</a>!</p>
<p>As most of you know, Apache Mesos has been running in production at some of the largest Internet companies like <a href="https://twitter.com">Twitter</a>, <a href="https://netflix.com">Netflix</a>, <a href="https://uber.com">Uber</a> and <a href="http://www.jpl.nasa.gov/">NASA JPL</a> for many years now. It has been proven to scale to tens of thousands of nodes, running hundreds of thousands of containers serving live traffic without interruption for months at a time.</p>
<p>Given its track record at these high profile companies, Apache Mesos could have reached its 1.0 milestone a long time ago. But one thing we really wanted to improve upon before doing a 1.0 release was to develop new HTTP APIs to ease the lives of both framework developers and cluster operators alike. These new APIs make it really easy to write a new framework or develop a new tool with off-the-shelf HTTP libraries written in the language of your choice. After iterating on the new APIs for nearly a year, we are confident that they are now ready for widespread use. Combined with a versioning scheme for the new APIs and a well-formulated <a href="http://mesos.apache.org/documentation/latest/versioning/">release and support process</a>, we are happy to call our latest release of Apache Mesos 1.0.</p>
<p>In addition to the new APIs, the 1.0 release is jam-packed with features that makes running stateless and stateful workloads easy. Below, we will discuss some of the highlights in this release. For further details please refer to the <a href="https://github.com/apache/mesos/blob/1.0.0/CHANGELOG">CHANGELOG</a>.</p>
<h3>HTTP API</h3>
<p>Before 1.0, there were essentially two types of APIs exposed by Mesos – the driver-based Framework API used by schedulers and executors and the REST-based HTTP API used by operators and tools. While the driver-based API made writing frameworks easier by hiding the complexities of interfacing with Mesos itself (e.g., leader detection, reliable reconnection, message delivery, etc.) it had several drawbacks. For example, only a few languages had bindings for the driver (specifically, C++, Java and Python), limiting the languages that frameworks could be written in. Moreover, the driver required both the client (e.g., scheduler) and the server (e.g., master) to open connections to one another in order to communicate. This requirement for a bi-directional connection between clients and servers made it really hard for clients to run inside containers or behind firewalls.</p>
<p>The new <a href="https://www.youtube.com/watch?v=mDn_iyFSv38&index=13&list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC">HTTP API</a> aims to solve these problems by providing an RPC-based HTTP API for both frameworks (i.e. <a href="http://mesos.apache.org/documentation/latest/scheduler-http-api/">schedulers</a> and <a href="http://mesos.apache.org/documentation/latest/executor-http-api/">executors</a>) and <a href="http://mesos.apache.org/documentation/latest/operator-http-api/">operators</a>. Since the API client libraries can be written in any language that speaks HTTP, it immediately opens the door for frameworks to be written in new languages. More importantly, the HTTP API doesn’t require servers to open a connection back to the client to communicate with it. As such, robust strategies for handling network partitions can now be implemented easily within the Mesos codebase itself.</p>
<p>Apart from simplifying the API for framework developers and operators, one of the most exciting new features in the new HTTP API is <strong>experimental</strong> support for <a href="http://mesos.apache.org/documentation/latest/operator-http-api/">event streams</a>. Event streams are a frequently requested – and long awaited feature – that have finally made their way into the Mesos codebase. Instead of continuously polling the heavy weight /state endpoint, clients (e.g, service discovery systems) can now get events streamed to them directly by the master.</p>
<h3>Unified Containerizer</h3>
<p>Before the unified containerizer support was available, Mesos depended on external daemons to run containers based on a specific image format; <code>docker</code> daemon to run docker images, <code>rkt</code> to run appc images and so on. This made the system architecture complex and added more failure points to the platform. With the <a href="https://www.youtube.com/watch?v=rHUngcGgzVM&index=14&list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC">unified containerizer</a>, we can now use a single container runtime engine, the <code>MesosContainerizer</code> to run containers off the most popular image formats (Docker, Appc, and soon OCI) without dependency on any external daemons. This makes the platform extremely robust, simplifies the system architecture and gives a unified isolation model for containers, allowing developers to achieve higher feature velocity in improving Mesos.</p>
<h3>Networking</h3>
<p>With the addition of support for the Container Networking Interface (<a href="https://github.com/containernetworking/cni">CNI</a>) in 1.0, we have made significant strides towards a comprehensive networking story in Mesos. Now, each container gets its own network namespace, which provides network isolation and allows users to work with containers as if they were running on bare metal or VM. Without network isolation, users have to manage network resources such as TCP/UDP ports on the agent. Managing these resources complicates the design of applications significantly.</p>
<p>One of the highlights of the new CNI support is the ability to now provide a single IP per container in Mesos. This is one of the most requested features that Mesos has ever had!! Since the CNI interface is widely supported by networking vendors, this gives Mesos tremendous flexibility in orchestrating containers on a wide variety of networking technologies (e.g, VxLAN, <a href="https://dcos.io/docs/1.8/administration/overlay-networks/">DC/OS overlay</a>, <a href="https://www.projectcalico.org/">Calico</a>, <a href="https://github.com/weaveworks/weave-cni">Weave</a>, <a href="https://github.com/containernetworking/cni/blob/master/Documentation/flannel.md">Flannel</a>).</p>
<h3>Storage</h3>
<p>Starting from Mesos 1.0, we added <strong>experimnental support</strong> for external storage to Mesos. Before this feature, while users could use persistent volumes for running stateful services, there were some limitations. First, the users were not able to easily use non-local storage volumes. Second, data migrations for local persistent volumes had to be manually handled by operators. The newly added <code>docker/volume</code> isolator addresses these limitations. Currently, the isolator interacts with the Docker volume plugins (e.g., <a href="https://github.com/emccode/rexray">REX-Ray</a>, <a href="https://github.com/ClusterHQ/flocker">Flocker</a>, <a href="https://github.com/rancher/convoy">Convoy</a>) using a tool called <code>dvdcli</code>. By speaking the Docker volume plugin API, Mesos containers can connect with external volumes from numerous storage providers (e.g., Amazon EBS, Ceph, EMC ScaleIO).</p>
<h3>Security</h3>
<p>Mesos 1.0 comes with significant improvements in authentication and authorization. While we have long supported framework authentication and SSL/TLS encryption for frameworks and REST endpoints, not all HTTP endpoints were authenticated or authorized. With 1.0, all the sensitive endpoints have support for authentication and authorization.</p>
<p>More importantly, we have added foundations for multi-tenancy by adding <a href="https://www.youtube.com/watch?v=-yWHuxXwuAA&index=20&list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC">fine-grained authorization</a> controls. For example, it is now possible to set up ACLs so that a user can only view information about her own tasks in the WebUI and/or HTTP endpoints. This is a must-have for enterprises that need to enforce strict policies around access to information. As part of this effort, the Authorizer interface has been completely overhauled to make it easy for module writers to plugin enterprise-specific security policies.</p>
<h3>GPU Support</h3>
<p>That’s right, you can now run your <a href="https://www.tensorflow.org/">TensorFlow</a> jobs <a href="https://www.youtube.com/watch?v=giJ4GXFoeuA&list=PL903xBaKIoaWc0hnSVyAAGAJUPxq2MCd0">directly on Mesos</a>. The custom GPU isolator handles all of the details of allocating GPUs to your application and making sure they are isolated from other jobs running on the system. All you have to worry about is making sure your newest deep learning algorithm is ready for deployment!</p>
<p>As a bonus, we mimic the functionality of <a href="https://github.com/NVIDIA/nvidia-docker">nvidia-docker</a> so you can take your existing GPU docker containers and run them on Mesos without modification. This means you can test your applications locally on docker and be assured that they will run as expected once you deploy them on Mesos.</p>
<p>We are working closely with the <a href="http://spark.apache.org/">Spark</a>, <a href="https://mesosphere.github.io/marathon/">Marathon</a>, and <a href="http://aurora.apache.org/">Aurora</a> teams to make scheduling your GPU workloads as easy as possible. We’ll keep you posted as more things develop!</p>
<h3>Windows Support</h3>
<p>Mesos 1.0 comes with <strong>experimental</strong> support for running Mesos Agent on Windows. It is now possible to build Mesos Agent and all its dependencies on Windows using CMake and Microsoft Visual Studio compiler. We are actively working on a tighter integration of Mesos Agent with Windows Container API (scheduled for release later this year), as well as adding support for launching Docker containers. The recent <a href="https://www.youtube.com/watch?v=bbyK1EBxrek">Mesoscon talk</a> gives an overview of the current state of this work as well as presents the roadmap for Mesos Agent on Windows.</p>
<h3>Community</h3>
<p>We would like to take this opportunity to give a huge thank you to the incredible Mesos community, without which none of this would be possible. We are fortunate to have a dedicated and enthusiastic community that not only finds and reports bugs but contributes code. In fact, all the highlighted features described above were done in close collaboration with several independent members and organizations in the community (e.g, IBM, Microsoft, Nvidia). It is rewarding to see an ever growing list of organizations using Mesos in production at scale and <a href="http://mesos.apache.org/blog/dev-community-status/">contributing back to the community</a>. Some of these companies have not only built frameworks but entire platforms (e.g., <a href="https://azure.microsoft.com/en-us/documentation/videos/azurecon-2015-deep-dive-on-the-azure-container-service-with-mesos/">Azure Container Service</a>, <a href="https://dcos.io/">DC/OS</a>, <a href="http://rancher.com/mesos/">Rancher</a>) on top of Mesos, which is truly humbling.</p>
<p>If you are new to Mesos and interested in learning more, please <a href="http://mesos.apache.org/community/">join us</a> on this incredible journey. We are nowhere close to done implementing all the things that we want to build in Mesos and we could always use more help.</p>
http://mesos.apache.org/blog/dev-community-status/
Mesos Developer Community Status Report
2016-07-21T00:00:00+00:00Michael Park
<p>The Mesos project has a large and rapidly growing community of users and contributors.
In this post we would like to reflect on the current state
of the community as well as speak about some benefits and challenges that the
growth brings.</p>
<p>One of such challenges is that the we have to make sure that everyone’s interests
are met. This isn’t always easy due to occasional conflicts of interest. For example,
developers are eager to see their patches vetted and committed to the codebase as
rapidly as possible, whereas production users of Mesos want to make sure that the stability of the software is never
compromised for the sake of rolling out new features. Both are essential components
of a healthy open source infrastructure project, and we want to strike a harmonizing
balance.</p>
<p>Transparency is the key for being able to grow the community while also making
sure that participants understand and agree with the development direction. In
this post - much of which derives from a recent <a href="https://www.youtube.com/watch?list=PLGeM09tlguZQVL7ZsfNMffX9h1rGNVqnC&v=SnPmU61fVjQ">MesosCon 2016 presentation</a> -
we are going to present some statistics that reflect the current state of the
community participation. The time frame for the following statistics ranges from
the 0.24 release (July 2015) until 1.0.0-rc2 (June/July 2016).</p>
<p>We mined the data for this post from Mesos <a href="https://git1-us-west.apache.org/repos/asf?p=mesos.git;a=tree;f=docs;h=b8c6cfa978ea6986fa7e9181f209448f4984a3ab;hb=9c8bfa9b1bfe52fa6b44aaf883333311bdde5519">git log</a> and used the recently added
<a href="https://git1-us-west.apache.org/repos/asf?p=mesos.git;a=blob;f=docs/contributors.yaml;h=e17fed49ea358837eb33827dc3aceea9cd69c1b3;hb=9c8bfa9b1bfe52fa6b44aaf883333311bdde5519">contributors.yaml</a> file for mapping contributors to organizations.</p>
<h3>Community Growth and Diversity</h3>
<p><%= image_tag ‘blog/2016-developer-community-1.png’ %></p>
<p>The number of unique contributors has been increasing steadily over time. As of
this writing, the number of contributors for the most recent release is greater
than 100. This is a more than factor two improvement over the 0.24 release for
which there were about 50 unique contributors!</p>
<p>Historically there have always been large organizations from which the significant
amount of contributions came from. In recent times, Mesosphere, IBM and Microsoft have
been leading in terms of number of contributions made to the codebase. However,
in terms of numbers of people, the individual contributors greatly outnumber those
from Mesosphere, IBM and Microsoft.</p>
<p>This is a great milestone for all of our individual contributors out there!
We greatly appreciate your contribution to the project and we’re looking forward
to seeing even more commits from individual contributors in the future.</p>
<p><%= image_tag ‘blog/2016-developer-community-2.png’ %></p>
<p>In terms of numbers of commits from an organization, Mesosphere has the highest number after
contributing 62.8% of the commits. It is followed by ‘Other’ (which is a default organization
for all the authors who do not have an affiliation entry in the <a href="https://git1-us-west.apache.org/repos/asf?p=mesos.git;a=blob;f=docs/contributors.yaml;h=e17fed49ea358837eb33827dc3aceea9cd69c1b3;hb=9c8bfa9b1bfe52fa6b44aaf883333311bdde5519">contributors.yaml</a>), IBM and Microsoft. We are actively working on involving more members from the community
and are looking forward to seeing an even greater community participation.</p>
<p>It is important to note that 1.0 release has been in the making for a longer time
than previous releases, and that is why there is a significant spike in the number of
commits for 1.0-rc2.
Here is how the same graph looks like once we change the breakdown from releases to months.</p>
<p><%= image_tag ‘blog/2016-developer-community-3.png’ %></p>
<p>It presents a fuller picture where the upward trend is still there, but one can also notice
that the smooth growth trend has been there for a while.</p>
<p>The last graph that we want to present shows the breakdown of commits per release
between contributors and the PMC members (aka Mesos Committers). As one can see
the number of commits authored by non-committers vastly outnumbers the ones originating
from committers. This shows that the active committers spend a lot of time shepherding
and landing patches from contributors.</p>
<p><%= image_tag ‘blog/2016-developer-community-4.png’ %></p>
<p>Going forward we are going to introduce a page on our website where we will continuously
publish up-to-date statistics around the developer community. We are also going to double
down on initiatives to improve the contribution experience on the Mesos project. We
started by doing a spring cleaning on our GitHub and Reviewboard backlogs, as well as the Kanban
board in JIRA. This will make sure that shepherds do not get drowned in stale reviews, and that
everyone can see what the community is working on. We also have some ideas around improving the
developer tools, collecting and analyzing metrics around reviews, as well as some other initiatives
that should foster a greater participation. If you are interested in helping us improve the
community participation and drive some of these initiatives please join the ‘Community’ working group.</p>
<p>Stay tuned for more news on the dev list!</p>
http://mesos.apache.org/blog/mesos-0-27-3-released/
Apache Mesos 0.27.3 Released
2016-06-19T00:00:00+00:00Jie Yu
<p>The latest Mesos point release for 0.27, 0.27.3, is now available for
<a href="http://mesos.apache.org/downloads">download</a>. This release includes fixes and
improvements for the following:</p>
<ul>
<li>[MESOS-4705] - Linux ‘perf’ parsing logic may fail when OS distribution has perf backports.</li>
<li>[MESOS-4869] - /usr/libexec/mesos/mesos-health-check using/leaking a lot of memory.</li>
<li>[MESOS-5018] - FrameworkInfo Capability enum does not support upgrades.</li>
<li>[MESOS-5021] - Memory leak in subprocess when ‘environment’ argument is provided.</li>
<li>[MESOS-5449] - Memory leak in SchedulerProcess.declineOffer.</li>
</ul>
<p>Full release notes are available in the release
<a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.3">CHANGELOG</a>.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 0.27.2 cluster to Mesos 0.27.3 are
straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade
guide</a> for detailed
information on upgrading to Mesos 0.27.3.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think. If you
run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and
IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 4 contributors who made 0.27.3 possible:</p>
<p>Fan Du, Vinod Kone, Benjamin Mahler, Dario Rexin, Yongqiao Wang, Jie Yu</p>
http://mesos.apache.org/blog/mesos-0-28-2-released/
Apache Mesos 0.28.2 Released
2016-06-13T00:00:00+00:00Jie Yu
<p>The latest Mesos release, 0.28.2, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes some important bug fixes and improvements on top of 0.28.1. It is highly recommended that users use this version if you are considering using Mesos 0.28. More specifically, this release includes the following fixes and improvements:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4705">MESOS-4705</a> - Linux ‘perf’ parsing logic may fail when OS distribution has perf backports.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5239">MESOS-5239</a> - Persistent volume DockerContainerizer support assumes proper mount propagation setup on the host.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5253">MESOS-5253</a> - Isolator cleanup should not be invoked if they are not prepared yet.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5282">MESOS-5282</a> - Destroy container while provisioning volume images may lead to a race.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5312">MESOS-5312</a> - Env <code>MESOS_SANDBOX</code> is not set properly for command tasks that changes rootfs.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4885">MESOS-4885</a> - Unzip should force overwrite.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5449">MESOS-5449</a> - Memory leak in SchedulerProcess.declineOffer.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5380">MESOS-5380</a> - Killing a queued task can cause the corresponding command executor to never terminate.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5307">MESOS-5307</a> - Sandbox mounts should not be in the host mount namespace.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.28.2">CHANGELOG</a></p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 0.28.1 cluster to Mesos 0.28.2 are straightforward. Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 0.28.2.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think. If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 10 contributors who made 0.28.2 possible:</p>
<p>Fan Du, Tomasz Janiszewski, Vinod Kone, Shuai Lin, Ben Mahler, Dario Rexin, Silas Snider, Gilbert Song, Yongqiao Wang, Jie Yu</p>
http://mesos.apache.org/blog/mesos-0-26-1-and-more-released/
Apache Mesos 0.26.1 and More Released
2016-04-12T00:00:00+00:00Michael Park
<p>The latest Mesos 0.26.1, 0.25.1 and 0.24.2 are now available for <a href="/downloads">download</a>.
The applicable subset of the following issues were backported.</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-920">MESOS-920</a> - GLOG performance.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-3046">MESOS-3046</a> - UUID performance.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-3051">MESOS-3051</a> - Port Ranges performance.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-3052">MESOS-3052</a> - Allocator filter performance.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-3307">MESOS-3307</a> - Configurable task/framework history.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-3560">MESOS-3560</a> - JSON-based credential files.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-3602">MESOS-3602</a> - HDFS.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-3738">MESOS-3738</a> - Mesos health check within docker container.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-3773">MESOS-3773</a>, <a href="https://issues.apache.org/jira/browse/MESOS-4069">MESOS-4069</a> - SSL.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-3834">MESOS-3834</a> - Agent upgrade compatibility</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4106">MESOS-4106</a> - Health checks.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4235">MESOS-4235</a> - <code>/state</code> endpoint performance.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4302">MESOS-4302</a> - Offer filter timeout fix for backlogged allocator.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4687">MESOS-4687</a> - Fixed point resources math.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4711">MESOS-4711</a> - Libevent.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4979">MESOS-4979</a> - Deletion of special files.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-5021">MESOS-5021</a> - Memory leak in subprocess.</li>
</ul>
<p>Full release notes are available in the release <a href="https://github.com/apache/mesos/blob/master/CHANGELOG">CHANGELOG</a>.</p>
<p>Upgrading to 0.26.1 can be done seamlessly on a 0.26.0 cluster. If upgrading from an earlier version, please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrades</a> documentation.</p>
<h2>Contributors</h2>
<p>Thanks to the 20 contributors who made 0.26.1, 0.25.1, and 0.24.2 possible:</p>
<p>Alexander Rojas, Alexander Rukletsov, Anand Mazumdar, Benjamin Mahler, Felix Abecassis, Gilbert Song, haosdent huang, Isabel Jimenez, James Peach, Joerg Schad, Jojy Varghese, Joris Van Remoortere, Joseph Wu, Kapil Arya, Kevin Klues, Klaus Ma, Michael Park, Neil Conway, Till Toenshoff, Timothy Chen.</p>
http://mesos.apache.org/blog/mesos-0-28-0-released/
Apache Mesos 0.28.0 Released
2016-03-17T00:00:00+00:00Vinod Kone
<p>The latest Mesos release, 0.28.0, is now available for <a href="http://mesos.apache.org/downloads">download</a>.</p>
<p>This release includes the following features and improvements:</p>
<ul>
<li><p>[MESOS-4343] - A new cgroups isolator for enabling the net_cls subsystem in
Linux. The cgroups/net_cls isolator allows operators to provide network
performance isolation and network segmentation for containers within a Mesos
cluster. To enable the cgroups/net_cls isolator, append <code>cgroups/net_cls</code> to
the <code>--isolation</code> flag when starting the slave. Please refer to
docs/mesos-containerizer.md for more details.</p></li>
<li><p>[MESOS-4687] - The implementation of scalar resource values (e.g., “2.5
CPUs”) has changed. Mesos now reliably supports resources with up to three
decimal digits of precision (e.g., “2.501 CPUs”); resources with more than
three decimal digits of precision will be rounded. Internally, resource math
is now done using a fixed-point format that supports three decimal digits of
precision, and then converted to/from floating point for input and output,
respectively. Frameworks that do their own resource math and manipulate
fractional resources may observe differences in roundoff error and numerical
precision.</p></li>
<li><p>[MESOS-4479] - Reserved resources can now optionally include “labels”.
Labels are a set of key-value pairs that can be used to associate metadata
with a reserved resource. For example, frameworks can use this feature to
distinguish between two reservations for the same role at the same agent
that are intended for different purposes.</p></li>
<li><p>[MESOS-2840] - <strong>Experimental</strong> support for container images in Mesos
containerizer (a.k.a. Unified Containerizer). This allows frameworks to
launch Docker/Appc containers using Mesos containerizer without relying on
docker daemon (engine) or rkt. The isolation of the containers is done using
isolators. Please refer to docs/container-image.md for currently supported
features and limitations.</p></li>
<li><p>[MESOS-4793] - <strong>Experimental</strong> support for v1 Executor HTTP API. This
allows executors to send HTTP requests to the /api/v1/executor agent
endpoint without the need for an executor driver. Please refer to
docs/executor-http-api.md for more details.</p></li>
<li><p>[MESOS-4370] Added support for service discovery of Docker containers that
use Docker Remote API v1.21.</p></li>
</ul>
<p>Additional API Changes:</p>
<ul>
<li>[MESOS-4066] - Agent should not return partial state when a request is made to /state endpoint during recovery.</li>
<li>[MESOS-4547] - Introduce TASK_KILLING state.</li>
<li>[MESOS-4712] - Remove ‘force’ field from the Subscribe Call in v1 Scheduler API.</li>
<li>[MESOS-4591] - Change the object of ReserveResources and CreateVolume ACLs to <code>roles</code>.</li>
<li>[MESOS-3583] - Add stream IDs for HTTP schedulers.</li>
<li>[MESOS-4427] - Ensure ip_address in state.json (from NetworkInfo) is valid</li>
</ul>
<p>Furthermore, several bugfixes and improvements made it into this release.
For full release notes with all features and bug fixes, please refer to the <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.28.0">CHANGELOG</a>.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 0.27.0 cluster to Mesos 0.28.0 are straightforward. There are just some minor, backwards compatible deprecations.
Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 0.28.0.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think.
If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 44 contributors who made 0.28.0 possible:</p>
<p>Abhishek Dasgupta,Alex Clemmer,Alex Naparu,Alexander Rojas,Alexander Rukletsov,Anand Mazumdar,Avinash sridharan,Benjamin Bannier,Benjamin Mahler,Bernd Mathiske,Cong Wang,Daniel Pravat,David Forsythe,Diana Arroyo,Disha Singh,Gilbert Song,Greg Mann,Guangya Liu,Isabel Jimenez,James Peach,Jan Schlicht,Jie Yu,Joerg Schad,Jojy Varghese,Joris Van Remoortere,Joseph Wu,Kapil Arya,Kevin Devroede,Kevin Klues,Klaus Ma,M Lawindi,Michael Browning,Michael Lunøe,Michael Park,Neil Conway,Shuai Lin,Till Toenshoff,Timothy Chen,Vinod Kone,Yong Tang,Yongqiao Wang,Zhiwei Chen,haosdent huang,mlawindi</p>
http://mesos.apache.org/blog/mesos-0-27-2-released/
Apache Mesos 0.27.2 Released
2016-03-07T00:00:00+00:00Michael Park
<p>The latest Mesos release, 0.27.2, is now available for <a href="http://mesos.apache.org/downloads">download</a>.
This release includes fixes and improvements for the following:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4693">MESOS-4693</a> - Variable shadowing in HookManager::slavePreLaunchDockerHook.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4711">MESOS-4711</a> - Race condition in libevent poll implementation causes crash.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4754">MESOS-4754</a> - The “executors” field is exposed under a backwards incompatible schema.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4687">MESOS-4687</a> - Implement reliable floating point for scalar resources.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.2">CHANGELOG</a>.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 0.27.1 cluster to Mesos 0.27.2 are straightforward.
Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 0.27.2.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think.
If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 4 contributors who made 0.27.2 possible:</p>
<p>Alexander Rojas, Kevin Devroede, Michael Park, Neil Conway</p>
http://mesos.apache.org/blog/mesos-0-27-1-released/
Apache Mesos 0.27.1 Released
2016-02-22T00:00:00+00:00Michael Park
<p>The latest Mesos release, 0.27.1, is now available for <a href="http://mesos.apache.org/downloads">download</a>.
This release includes fixes and improvements for: reconnection logic for Zookeeper client, <code>/state</code> endpoint and <code>systemd</code> integration.</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4546">MESOS-4546</a> - Mesos Agents needs to re-resolve hosts in zk string on leader change / failure to connect.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-4582">MESOS-4582</a> - state.json serving duplicate “active” fields.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-3007">MESOS-3007</a> - Support systemd with Mesos.</li>
</ul>
<p>Full release notes are available in the release <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.1">CHANGELOG</a>.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 0.27.0 cluster to Mesos 0.27.1 are straightforward.
Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 0.27.1.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think.
If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 8 contributors who made 0.27.1 possible:</p>
<p>Jie Yu, Joerg Shad, Joris Van Remoortere, Joseph Wu, Kapil Arya, Michael Park, Neil Conway, Shuai Lin</p>
http://mesos.apache.org/blog/mesoscon-2016-cfp-is-now-open/
MesosCon 2016 CFP is now open!
2016-02-12T00:00:00+00:00Michael Park
<p>MesosCon North America 2016 will be held in Denver, CO on June 1-2, 2016.</p>
<p>Talk submissions, sponsorship opportunities, and early-bird registration are now open for the conference.</p>
<h2>Speak at MesosCon</h2>
<p>Several formats are being accepted for speaking proposals, including:
Presentations, Panels, Keynotes and Lightning Talks.
Submissions are being accepted through <strong>March 9, 2016</strong>.</p>
<p>You can take a look at <a href="https://www.youtube.com/playlist?list=PLVjgeV_avap2arug3vIz8c6l72rvh9poV">MesosCon 2015 list of talks</a>
for ideas. If you’re unsure about your proposal, or want some feedback or advice in general,
please don’t hesitate to reach out to the <a href="https://mail-archives.apache.org/mod_mbox/mesos-dev">mesos-dev</a> mailing list.
We’ll be happy to help out! Further details are available on the <a href="http://events.linuxfoundation.org/events/mesoscon/program/cfp">CFP website</a>.</p>
<p>All submissions for MesosCon North America will also be considered for the upcoming MesosCon Europe and China events (details to be announced shortly).</p>
<h2>Sponsor MesosCon</h2>
<p>The <a href="http://events.linuxfoundation.org/events/mesoscon/sponsors">sponsorship prospectus for MesosCon</a> is now available,
including several opportunities with limited availability.</p>
<h2>Attend MesosCon</h2>
<p><a href="http://events.linuxfoundation.org/events/mesoscon/attend/register">Early-Bird registration</a> is open through April 22nd!</p>
<p>We look forward to seeing you at MesosCon North America, as well as MesosCon Europe and MesosCon China taking place later this year!</p>
http://mesos.apache.org/blog/mesos-0-27-0-released/
Apache Mesos 0.27.0 Released
2016-02-02T00:00:00+00:00Kapil Arya
<p>The latest Mesos release, 0.27.0, is now available for <a href="http://mesos.apache.org/downloads">download</a>.
This release includes the following features and improvements:</p>
<ul>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-1791">[MESOS-1791]</a>: Support for <em>resource quota</em> that provides non-revocable resource guarantees without tying reservations to particular Mesos agents. Please refer to <a href="http://mesos.apache.org/documentation/latest/quota">the quota documentation</a> for more information.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-191">[MESOS-191]</a>: <em>Multiple disk</em> support to allow for disk IO intensive applications to achieve reliable, high performance.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-4085">[MESOS-4085]</a>: Flexible roles with the introduction of <em>implicit roles</em>. It deprecates the whitelist functionality that was implemented by specifying <code>--roles</code> during master startup to provide a static list of roles.</p></li>
<li><p><a href="https://issues.apache.org/jira/browse/MESOS-2353">[MESOS-2353]</a>: Performance improvement of the state endpoint for large clusters.</p></li>
</ul>
<p>Furthermore, 167+ bugfixes and improvements made it into this release.
For full release notes with all features and bug fixes, please refer to the <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.27.0">CHANGELOG</a>.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 0.26.0 cluster to Mesos 0.27.0 are straightforward. There are just some minor, backwards compatible deprecations.
Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 0.27.0.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think.
If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 55 contributors who made 0.27.0 possible:</p>
<p>Adam B, Alexander Rojas, Alexander Rukletsov, Alex Clemmer, Anand Mazumdar, Andy Pang, Anindya Sinha, Artem Harutyunyan, Avinash sridharan, Bartek Plotka, Benjamin Bannier, Benjamin Hindman, Benjamin Mahler, Bernd Mathiske, Bhuvan Arumugam, BrickXu, Chi Zhang, Cong Wang, Dario Rexin, David Forsythe, Diana Arroyo, Diogo Gomes, Felix Abecassis, Gilbert Song, Greg Mann, Guangya Liu, haosdent huang, James Peach, Jan Schlicht, Jian Qiu, Jie Yu, Jihun Kang, Jiri Simsa, Jocelyn De La Rosa, Joerg Schad, Jojy Varghese, Joris Van Remoortere, Joseph Wu, Kapil Arya, Kevin Klues, Klaus Ma, Mandeep Chadha, Michael Park, Neil Conway, Olivier Sallou, Qian Zhang, Shuai Lin, Stephan Erb, Steve Hoffman, Till Toenshoff, Timothy Chen, Vaibhav Khanduja, Vinod Kone, Vivek Juneja, and Zhitao Li.</p>
http://mesos.apache.org/blog/mesos-0-26-0-released/
Apache Mesos 0.26.0 Released
2015-12-16T00:00:00+00:00Till Toenshoff
<p>The latest Mesos release, 0.26.0, is now available for <a href="http://mesos.apache.org/downloads">download</a>.
103+ bugfixes and improvements made it into this release.
For full release notes with all features and bug fixes, please refer to the <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.26.0">CHANGELOG</a>.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 0.25.0 cluster to Mesos 0.26.0 are straightforward. There are just some minor, backwards compatible deprecations.
Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 0.26.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think.
If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 45 contributors who made 0.26.0 possible:</p>
<p>Adam B, Alex Clemmer, Alexander Rojas, Alexander Rukletsov, Anand Mazumdar, Andrey Dyatlov, Artem Harutyunyan, Avinash sridharan, Benjamin Bannier,
Benjamin Hindman, Benjamin Mahler, Bernd Mathiske, Bhuvan Arumugam, Chengwei Yang, Chi Zhang, Connor Doyle, Dave Lester, Elsmore, Felix Abecassis,
Gastón Kleiman, Gilbert Song, Greg Mann, Guangya Liu, haosdent huang, Isabel Jimenez, James Peach, Jian Qiu, Jie Yu, Joerg Schad, Jojy Varghese,
Jonathon Rossi, Joris Van Remoortere, Joseph Wu, Kapil Arya, Klaus Ma, Marco Massenzio, Michael Park, Neil Conway, Rossi, Spike Curtis,
Till Toenshoff, Timothy Chen, Tomasz Janiszewski, Vinod Kone, Yong Qiao Wang</p>
http://mesos.apache.org/blog/mesos-0-25-0-released/
Apache Mesos 0.25.0 Released
2015-10-12T00:00:00+00:00Niklas Q. Nielsen
<p>The latest Mesos release, 0.25.0, is now available for <a href="http://mesos.apache.org/downloads">download</a>.
This release includes the following features and improvements:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1474">[MESOS-1474]</a> - Experimental support for maintenance primitives. Please refer to <a href="http://mesos.apache.org/documentation/latest/maintenance/">the maintenance documentation</a> for more information.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2600">[MESOS-2600]</a> - Added master endpoints /reserve and /unreserve for dynamic reservations. Please refer to <a href="http://mesos.apache.org/documentation/latest/reservation/">the reservation documentation</a> for more information.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2044">[MESOS-2044]</a> - Extended Module APIs to enable IP per container assignment, isolation and resolution. Please refer to <a href="http://mesos.apache.org/documentation/latest/networking-for-mesos-managed-containers/">the container networking documentation</a> for more information.</li>
</ul>
<p>Furthermore, 100+ bugs and improvements have made it into this release.
For full release notes with all features and bug fixes, please refer to the <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.25.0">CHANGELOG</a>.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 0.24.x cluster to Mesos 0.25 are straightforward, but there are a few caveats/deprecations.
Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 0.25.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think.
If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 40 contributors who made 0.25.0 possible:
Adam B, Aditi Dixit, Alex Clemmer, Alexander Rukletsov, Anand Mazumdar, Artem Harutyunyan, Ben Mahler, Bernd Mathiske, Chi Zhang, Cong Wang, Eijsermans, Eren Güve, Felix Abecassis, Greg Mann, Guangya Liu, Isabel Jimenez, James Peach, Jan Schlicht, Jiang Yan Xu, Jie Yu, Joerg Schad, Jojy Varghese, Joris Van Remoortere, Joseph Wu, Kapil Arya, Klaus Ma, Lily Chen, M Bauer, Marco Massenzio, Michael Park, Neil Conway, Niklas Nielsen, Paul Brett, Qian Zhang, Timothy Chen, Vaibhav Khanduja, Vinod Kone, Wojciech Sielski, Yong Qiao Wang and Haosdent Huang</p>
http://mesos.apache.org/blog/announcing-mesoscon-eu-2015-hackathon/
Announcing the #MesosCon Europe Hackathon
2015-09-30T00:00:00+00:00Casey Bisson
<p>We are proud to announce the first ever <a href="http://events.linuxfoundation.org/events/mesoscon-europe">MesosCon Europe</a> hackathon on Friday, October 9th, 2015. This event is an opportunity to contribute to the project, learn from other members of the community, and build some fantastic hacks.</p>
<p>The hackathon is being sponsored by <a href="https://www.joyent.com/">Joyent</a>, and, though not required, participants are invited to use Joyent’s cloud during the hack day. Joyent is offering each participant substantial service credits in their cloud to kickstart work and continue hacking after the event.</p>
<h3>Award categories</h3>
<p>We are challenging participants to hack on Mesos in two categories:</p>
<ul>
<li><em>Community need</em>: These are issues logged in the Mesos issue tracker that have received a large amount of activity (comments, votes, etc), and address some of the larger problems or pain points for the community. Suggested projects include new features, bugs, or documentation.</li>
<li><em>People’s choice</em>: These contributions will be voted on by your peers, and our guidance is intentionally open-ended so teams will run wild with their ideas. We’re excited to see what you come up with.</li>
</ul>
<p>Additionally, Joyent is offering a <em>container-native bonus</em> award to one hack in each of the above categories that best leverages Joyent’s Triton bare metal container infrastructure. Joyent’s judges may award the container-native bonus at their discretion.</p>
<h3>Rules</h3>
<ul>
<li>Coding may not begin until the hackathon officially begins, although we encourage you to plan and scheme ahead of time!</li>
<li>Sourcecode from the hackathon must be fully available on Github, Bitbucket, or a public source repo in order to be successfully submitted. Code must additionally be licensed in a manner compatible with Mesos' licensing.</li>
<li>Submitted applications must run.</li>
<li>Teams may have a maximum of four contributors.</li>
<li>A team may only win one category.</li>
<li>Submissions must be related to Mesos in some way.</li>
</ul>
<p>These rules are subject to change. Please hack the code, not the rules: if you think you’re doing something that’s against the rules or that gives you an unfair advantage, you probably shouldn’t do it.</p>
<h3>Schedule</h3>
<p>9:00 - Hacking begins at conference center</p>
<p>13:00 - Lunch arrives</p>
<p>17:00 - Coding stops, refreshments are served</p>
<p>17:15 - Re-explain the voting rules</p>
<p>17:20 - Presentations start - Each team gets 5 minutes to pitch what they built and show it off</p>
<p>18:00 - Voting occurs</p>
<p>18:15 - Winners announced</p>
<h3>Registration</h3>
<p>The hackathon is open to all <a href="http://events.linuxfoundation.org/events/mesoscon-europe">MesosCon Europe 2015</a> attendees. Please indicate your interest in the hackathon while registering for the conference or update your registration to reflect it.</p>
http://mesos.apache.org/blog/mesos-0-24-1-and-more-released/
Apache Mesos 0.24.1 and More Released
2015-09-24T00:00:00+00:00Adam B
<p>The latest Mesos 0.24.1 is now available for <a href="/downloads">download</a>. This release includes a fix for version parsing for Docker 1.8 and 1.7.0.fc22, as well as a fix for Docker command health checks. These same fixes were backported onto 0.23.0 to produce a new Mesos 0.23.1 release. MESOS-2986 was also backported to 0.21 and 0.22 to produce Mesos 0.21.2 and 0.22.2 (still VOTING). All of these releases are (or will be) available on the <a href="/downloads">downloads</a> page.</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2986">MESOS-2986</a> - Docker version output is not compatible with Mesos</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-3136">MESOS-3136</a> - COMMAND health checks with Marathon 0.10.0 are broken</li>
</ul>
<p>Full release notes are available in the release <a href="https://github.com/apache/mesos/blob/master/CHANGELOG">CHANGELOG</a>.</p>
<p>Upgrading to 0.24.1 can be done seamlessly on a 0.24.0 cluster. In fact, only the slaves need to be upgraded. If upgrading from an earlier version, please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrades</a> documentation.</p>
<h2>Contributors</h2>
<p>Special thanks to the lone code contributor for 0.24.1, 0.23.1, 0.22.2, and 0.21.2: haosdent huang. Our hero.</p>
http://mesos.apache.org/blog/mesos-0-24-0-released/
Apache Mesos 0.24.0 Released
2015-09-21T00:00:00+00:00Vinod Kone
<p>The latest Mesos release, 0.24.0, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes the following features and improvements:</p>
<h4>Scheduler HTTP API (<a href="https://issues.apache.org/jira/browse/MESOS-2288">MESOS-2288</a>)</h4>
<p>Mesos 0.24 provides <strong>experimental</strong> support for scheduler HTTP API. Framework schedulers can communicate with Mesos by sending HTTP POST requests to <code>/api/v1/scheduler</code> instead of depending on the native libmesos library. The endpoint accepts both JSON and Protobuf requests. Additionally, the master talks back to the scheduler using the same connection opened by the scheduler, mitigating communication issues in firewalled/NATed environments. Refer to the <a href="http://mesos.apache.org/documentation/latest/scheduler_http_api/">scheduler http api documentation</a> for more information.</p>
<p>Note that, this release only adds the support for scheduler HTTP API. Support for HTTP executor API is currently in the works and will be released soon!</p>
<h3>API Versioning (<a href="https://issues.apache.org/jira/browse/MESOS-3167">MESOS-3167</a>)</h3>
<p>As part of this release, we have also outlined the versioning scheme for Mesos HTTP API going forward. At a high level, the Mesos API (constituting Scheduler, Executor, Internal, Operator/Admin APIs) will have a version in the URL. The versioned URL will have a prefix of <code>/api/vN</code> where <code>N</code> is the version of the API. For simplicity, the stable version of the API will correspond to the major release version of Mesos. For example, v1 of the API will be supported by Mesos release versions 1.0.0, 1.4.0, 1.20.0 etc. Refer to the <a href="https://docs.google.com/document/d/1-iQjo6778H_fU_1Zi_Yk6szg8qj-wqYgVgnx7u3h6OU/edit#">versioning design document</a> for more information.</p>
<h3>Changelog</h3>
<p>Hundreds of other bug fixes/improvements are included in Mesos 0.24.0.
See the <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.24.0">CHANGELOG</a> for a full list of resolved JIRA issues.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 0.23.x cluster to Mesos 0.24 are straightforward, but there are a few caveats/deprecations.
Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 0.24.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think. If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 37 contributors who made 0.24.0 possible:</p>
<p>Adam B, Aditi Dixit, Alex Clemmer, Alexander Rojas, Alexander Rukletsov, Anand Mazumdar, Anindya Sinha, Artem Harutyunyan, Bartek Plotka, Benjamin Hindman, Benjamin Mahler, Bernd Mathiske, Chi Zhang, Chris Heller, Dave Lester, Greg Mann, Guangya Liu, Ian Downes, Isabel Jimenez, James DeFelice, James Peach, Jan Schlicht, Jiang Yan Xu, Jie Yu, Joerg Schad, Jojy Varghese, Joris Van Remoortere, Joseph Wu, Kapil Arya, Klaus Ma, Lily Chen, Marco Massenzio, Mark Wang, Michael Park, Michael Schenck, Niklas Nielsen, Paul Brett, Ryuichi Okumura, Shuai Lin, Till Toenshoff, Tim Anderegg, Timothy Chen, Vinod Kone, Yong Qiao Wang, ayouwei, haosdent huang, usultrared.</p>
http://mesos.apache.org/blog/mesoscon-europe-2015-schedule-announced/
MesosCon Europe Schedule and Diversity Scholarship Available
2015-09-17T00:00:00+00:00Michael Hausenblas
<p><a href="http://events.linuxfoundation.org/events/mesoscon-europe">MesosCon Europe 2015</a> will take place in Dublin, Ireland in three weeks, and we’re excited to announce the event schedule and availability of a diversity scholarship program.</p>
<h2>Schedule</h2>
<p>The program committee was overwhelmed by the number of submissions to the first European edition of MesosCon — 65 talks and tutorials have been submitted, you all rock — <a href="http://markmail.org/message/qf5dbzmugoh4gswf">we turned to you</a>, the community to help us select the most interesting and relevant talks. Given we had hard constraints concerning time and space we ended up selecting 14 talks and four lightning talks based on your overall ranking.</p>
<p>The talks range from introductory presentations that aim helping the onboarding of new Mesos community members to deep dives: in the afternoon we will split into two tracks to be able to accommodate more topics. Sessions will take place on Thursday, October 8th 2015. Check out the <a href="http://events.linuxfoundation.org/events/mesoscon-europe/program/schedule">schedule</a> yourself!</p>
<p>We’ll also be holding a Mesos hackathon on Friday October 9th, and look forward to seeing what members of the community build and dream up.</p>
<h2>Diversity Scholarship Program</h2>
<p>As important as the schedule might be, if you can’t be there in person, what gives? So, we’re repeating the successful offering of a <a href="http://events.linuxfoundation.org/events/mesoscon-europe/attend/scholarship">scholarship program</a>: it provides support to women, people of color, and people with disabilities who may not otherwise have the opportunity to attend MesosCon for financial reasons. <a href="https://docs.google.com/a/twitter.com/forms/d/1YavIUe3-xjjmqxqpE_QYbBkDz3z72iada568NyPxRRg/viewform">Apply</a> today and hope to see you in Dublin.</p>
http://mesos.apache.org/blog/mesoscon-goes-europe/
MesosCon goes Europe
2015-08-05T00:00:00+00:00Michael Hausenblas
<p>The Mesos <a href="http://mesos.apache.org/community/">community</a> has been growing steadily over the past years and on behalf of the #MesosCon Program Committee I’m excited and proud to be able to share the great news with you: MesosCon is now a world-wide event and this fall it comes to Europe. Organized by the Apache Mesos community, <a href="http://events.linuxfoundation.org/events/mesoscon-europe">#MesosCon Europe</a> brings together users and developers to share and learn about the project and its growing ecosystem.</p>
<p>MesosCon Europe 2015 will take place in Dublin, Ireland and feature a full day of sessions to learn more about the Mesos core, an ecosystem developed around the project, and related technologies. The program will include workshops to get started with Mesos, keynote speakers from industry leaders, and sessions led by adopters and contributors. Day two will feature a hackathon held at the conference.</p>
<p>Now is a good time to start preparing yourself for #MesosCon Europe: no matter if you want to contribute through a talk submission (see the <a href="http://events.linuxfoundation.org/events/mesoscon-europe/program/cfp">CfP</a> for details) or want to support us through a sponsorship. Stay tuned via <a href="http://twitter.com/mesoscon">@MesosCon</a> and mark these dates in your calendar:</p>
<ul>
<li>Early bird registration closes: 24 August 2015</li>
<li>CfP deadline: 25 August 2015</li>
<li>Schedule announced: 15 September 2015</li>
<li>Conference dates: 8 and 9 October 2015</li>
</ul>
<p>More info: <a href="http://events.linuxfoundation.org/events/mesoscon-europe">http://events.linuxfoundation.org/events/mesoscon-europe</a></p>
<p>Help us make this first European #MesosCon event a big success by spreading the word and <a href="http://events.linuxfoundation.org/events/mesoscon-europe/attend/register">register today</a>!</p>
http://mesos.apache.org/blog/mesos-0-23-0-released/
Apache Mesos 0.23.0 Released
2015-07-29T00:00:00+00:00Adam B
<p>The latest Mesos release, 0.23.0, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes the following features and improvements:</p>
<h4>Per-container network isolation (<a href="https://issues.apache.org/jira/browse/MESOS-1585">MESOS-1585</a>)</h4>
<p>Mesos 0.23 provides support for per-container network monitoring and isolation. Network isolation prevents a single container from exhausting the available network ports, consuming an unfair share of the network bandwidth or significantly delaying packet transmission for others. Network statistics for each active container are published through the /monitor/statistics.json endpoint on the slave. Network isolation is transparent for the majority of tasks running on a slave (those that bind to port 0 and let the kernel allocate their port). This feature is only available on Linux and requires a configure-time flag.
Refer to the <a href="http://mesos.apache.org/documentation/latest/network-monitoring/">network monitoring and isolation documentation</a> for more information.</p>
<h4>SSL (<a href="https://issues.apache.org/jira/browse/MESOS-910">MESOS-910</a>)</h4>
<p>Experimental support for SSL encryption of any libprocess communication via libevent. Encrypting traffic between the Mesos master and its slaves and frameworks is important for information security, as it prevents eavesdropping and impersonation. This feature requires a configure-time flag and will have some performance impact.
Refer to the <a href="http://mesos.apache.org/documentation/latest/mesos-ssl/">Mesos SSL documentation</a> for instructions on building and enabling SSL.</p>
<h4>Oversubscription (<a href="https://issues.apache.org/jira/browse/MESOS-354">MESOS-354</a>)</h4>
<p>Experimental support for launching tasks/executors on revocable resources. These resources can be revoked by Mesos at any time, causing the tasks using them to be throttled or preempted.</p>
<p>High-priority user-facing services are typically provisioned on large clusters for peak load and unexpected load spikes. Hence, for most of time, the provisioned resources remain underutilized. Oversubscription takes advantage of temporarily unused resources to execute best-effort tasks such as background analytics, video/image processing, chip simulations, and other low priority jobs.</p>
<p>Oversubscription adds two new slave components: a Resource Estimator and a Quality of Service (QoS) Controller, alongside extending the existing resource allocator, resource monitor, and mesos slave.
Refer to the <a href="http://mesos.apache.org/documentation/latest/oversubscription/">oversubscription documentation</a> for more information.</p>
<h4>Persistent volumes (<a href="https://issues.apache.org/jira/browse/MESOS-1554">MESOS-1554</a>)</h4>
<p>Experimental support for frameworks creating Persistent Volumes from disk resources. This enables stateful services such as HDFS and Cassandra to store their data within Mesos rather than having to resort to network-mounted EBS volumes or unmanaged disk resources that need to be placed in a well-known location.
Refer to the <a href="http://mesos.apache.org/documentation/latest/persistent-volume/">persistent volume documentation</a> for more information.</p>
<h4>Dynamic reservations (<a href="https://issues.apache.org/jira/browse/MESOS-2018">MESOS-2018</a>)</h4>
<p>Experimental support for frameworks dynamically reserving resources on specific slaves for their role. Rather than requiring an operator to specify a fixed, precalculated set of “static” reservations on slave startup, a framework can now reserve resources as they are being offered, without requiring a slave restart.
No breaking changes were introduced with dynamic reservation, which means the existing static reservation mechanism continues to be fully supported.
Refer to the <a href="http://mesos.apache.org/documentation/latest/reservation/">reservation documentation</a> for more information.</p>
<h4>Fetcher caching (<a href="https://issues.apache.org/jira/browse/MESOS-336">MESOS-336</a>)</h4>
<p>Experimental support for fetcher caching of executor/task binaries. The fetcher can be instructed to cache URI downloads in a dedicated directory for reuse by subsequent downloads. If the URI’s “cache” field has the value “true”, then the fetcher cache is in effect. If a URI is encountered for the first time (for a particular user), it is first downloaded into the cache, then copied to the sandbox directory from there. If the same URI is encountered again (for the same user), and a corresponding cache file is resident in the cache or still en route into the cache, then downloading is omitted and the fetcher proceeds directly to copying from the cache.
Refer to the <a href="http://mesos.apache.org/documentation/latest/fetcher/">fetcher documentation</a> for more information.</p>
<h4>NOTE: Experimental status</h4>
<p>SSL, Oversubscription, Persistent Volumes, Dynamic Reservations, and Fetcher Caching are all released as “experimental” features, meaning that they are feature complete, at least for some use case, but have not yet been tested in production environments. We welcome your feedback.</p>
<h3>Changelog</h3>
<p>Hundreds of other bug fixes/improvements are included in Mesos 0.23.0.
See the <a href="https://gitbox.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.23.0">CHANGELOG</a> for a full list of resolved JIRA issues.</p>
<h3>Upgrades</h3>
<p>Rolling upgrades from a Mesos 0.22.x cluster to Mesos 0.23 are straightforward, but there are a few caveats/deprecations.
Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to Mesos 0.23.</p>
<h3>Compiler version requirement</h3>
<p>Please note that Mesos 0.23.0 now requires gcc 4.8+ or clang 3.5+, so that we can take advantage of C++11 language features.</p>
<h3>Try it out</h3>
<p>We encourage you to try out this release and let us know what you think. If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks!</h3>
<p>Thanks to the 52 contributors who made 0.23.0 possible:
Aaron Bell, Adam B, Aditi Dixit, Akanksha Agrawal, Alexander Rojas, Alexander Rukletsov, Anand Mazumdar, Artem Harutyunyan, Bartek Plotka, Benjamin Hindman, Benjamin Mahler, Bernd Mathiske, Brendan Chang, Brian Wickman, Chi Zhang, Christos Kozyrakis, Cody Maloney, Cong Wang, Connor Doyle, Dave Lester, Dominic Hamon, Evelina Dumitrescu, Gajewski, Greg Mann, haosdent huang, Ian Babrou, Ian Downes, Isabel Jimenez, Itamar Ostricher, James Peach, Jan Schlicht, Jay Buffington, Jiang Yan Xu, Jie Yu, Joerg Schad, Jojy Varghese, Joris Van Remoortere, Kapil Arya, Marco Massenzio, Mark Wang, Michael Park, Nancy Ko, Niklas Q. Nielsen, Oliver Nicholas, Paul Brett, Ricardo Cervera-Navarro, Stan Teresen, Till Toenshoff, Timothy Chen, Vinod Kone, weitao zhou, and Zhiwei Chen</p>
http://mesos.apache.org/blog/register-for-mesoscon-seattle-save/
Register for MesosCon by 7/31 and Save $100
2015-07-28T00:00:00+00:00Steven Borrelli
<p><a href="http://www.mesoscon.org">MesosCon</a> is less than a month away, and the standard registration fee of $399 expires July 31, 2015. <a href="http://events.linuxfoundation.org/events/mesoscon/attend/register">Register today</a>. Beginning August 1, 2015 the registration fee will be $499. Students are eligible for a special registration fee of US$99 (Note that a valid student id will be required at onsite check-in).</p>
<p>MesosCon is an annual conference organized by the Apache Mesos community, bringing together users and developers to share and learn about the project and its growing ecosystem.</p>
<h2>This Year’s Conference</h2>
<p>This year MesosCon will take place August 20 - 21, 2015 at Sheraton Seattle in Seattle, WA, and features four tracks of talks for developers, operations and users. <a href="http://events.linuxfoundation.org/events/mesoscon/program/schedule">MesosCon Schedule</a></p>
<p>MesosCon’s Keynote speakers feature Adrian Crockroft (Battery Ventures), Benjamin Hindman (Mesosphere), Neha Narula (MIT), and Peter Bailis (Berkeley).</p>
<p>The Mesos community will once again hold a <a href="https://www.eventbrite.com/e/mesoscon-2015-pre-conference-hackathon-tickets-17752101012">day-long hackathon</a>, this year on Wednesday, August 19th and sponsored by SignalFx.</p>
<p>On behalf of the #MesosCon Program Committee, we couldn’t be more excited about this year’s conference and what this means for the open source project’s community. We hope that you’ll join us!</p>
http://mesos.apache.org/blog/mesoscon-seattle-preconference-and-social-activities/
Announcing MesosCon pre-conference and social activities
2015-07-13T00:00:00+00:00Dave Lester
<p>The countdown until <a href="http://www.mesoscon.org">#MesosCon</a> 2015 in Seattle continues: only 38 days until the event! <a href="http://events.linuxfoundation.org/events/mesoscon/attend/register">Register today</a>!</p>
<p>We’re pleased to announce several pre-conference and social activities that we hope will make this year’s conference even more enjoyable.</p>
<h2>MesosCon Hackathon, Wednesday August 19th</h2>
<p>Join us for a <a href="https://www.eventbrite.com/e/mesoscon-2015-pre-conference-hackathon-tickets-17752101012">pre-conference hackathon on Wednesday, August 19th</a>; space is limited so be sure to register today! The hackathon is sponsored by <a href="https://signalfx.com">SignalFx</a>, and hosted by <a href="http://www.moz.com">Moz</a> at their office in downtown Seattle a short distance from the Sheraton Seattle conference hotel. The hackathon will take place from 8:30-5:00pm.</p>
<p>What should you work on? It’s really up to you, as long as it’s Mesos-related. Here are a few broad categories of projects to consider.</p>
<ul>
<li>Frameworks (build your own Mesos framework or add features to an existing framework)</li>
<li>Mesos Core (squashing bugs, adding new features, or even proposing a design document)</li>
<li>Ops (tools to make it easier to operate a Mesos cluster; examples may include monitoring, logging, etc)</li>
</ul>
<p>Additional details about the hackathon are available on the <a href="https://www.eventbrite.com/e/mesoscon-2015-pre-conference-hackathon-tickets-17752101012">Eventbrite registration page</a>.</p>
<h2>Attendee Offsite Reception, Thursday Evening, August 20th</h2>
<p>Open to all #MesosCon attendees, join us at the <a href="http://www.empmuseum.org/">Experience Music Project</a> (EMP), a leading-edge nonprofit museum, dedicated to the ideas and risk-taking that fuel contemporary popular culture. Appetizers and drinks included as well as the opportunity to check out some of the current exhibits. Shuttles will be provided from the Sheraton to the EMP starting at 7:30pm and returning later in the evening.</p>
<h2>Mesos Evening Event, Wednesday, August 19th</h2>
<p>Twitter plans to host a Mesos event the night before the conference at their Seattle office. More details will be announced in the coming week, so stay tuned!</p>
<h2>About MesosCon</h2>
<p>MesosCon will take place August 20 - 21, 2015 at Sheraton Seattle in Seattle, WA with a hackathon taking place the day prior to the event. The standard attendee registration fee will be US$399 (Through July 31, 2015), and late registration US$499 beginning August 1, 2015. Students are eligible for a special registration fee of US$99 (Note that a valid student id will be required at onsite check-in).</p>
<p>Hope to see you at this year’s MesosCon!</p>
http://mesos.apache.org/blog/mesoscon-seattle-keynote-speakers-announced/
MesosCon Seattle 2015 Keynote Speakers Announced
2015-07-01T00:00:00+00:00Dave Lester
<p>The countdown until <a href="http://events.linuxfoundation.org/events/mesoscon">#MesosCon</a> has begun:</p>
<blockquote class="twitter-tweet" lang="en"><p lang="en" dir="ltr">Only 7 weeks until <a href="https://twitter.com/hashtag/MesosCon?src=hash">#MesosCon</a>! Register today and join us in Seattle <a href="http://t.co/bVJ9dL5CEO">http://t.co/bVJ9dL5CEO</a></p>— #MesosCon (@mesoscon) <a href="https://twitter.com/mesoscon/status/616320613922275332">July 1, 2015</a></blockquote>
<p>MesosCon is an annual conference organized by the Apache Mesos community, bringing together users and developers to share and learn about the project and its growing ecosystem. <a href="http://events.linuxfoundation.org/events/mesoscon/attend/register">Register now</a> and join us in Seattle this August!</p>
<p>Here’s a rundown of several conference announcements:</p>
<h2>Keynote Speakers</h2>
<p>We’ve curated a stellar line-up of invited keynote speakers, including:</p>
<ul>
<li>Adrian Crockroft (Battery Ventures)</li>
<li>Benjamin Hindman (Mesosphere)</li>
<li>Neha Narula (MIT)</li>
<li>Peter Bailis (Berkeley)</li>
</ul>
<p>You don’t want to miss these! We’ll also feature platinum sponsored keynotes from Cisco, Intel, and Mesosphere.</p>
<h2>Mesos Hackathon</h2>
<p>The Mesos community will once again hold a day-long hackathon, this year on Wednesday, August 19th and sponsored by SignalFx. More details will be announced soon including the location and registration; stay tuned and hold the date!</p>
<h2>Lightning Talks</h2>
<p>Want to present at MesosCon, but missed the call for full sessions? We’ve opened a <a href="http://events.linuxfoundation.org/events/mesoscon/program/lighting-talks">CFP for lightning talks</a>; the application deadline is July 15th.</p>
<h2>Diversity Scholarship Program</h2>
<p>The <a href="http://events.linuxfoundation.org/events/mesoscon/attend/scholarship">MesosCon scholarship program</a> provides support to women, people of color, and people with disabilities who may not otherwise have the opportunity to attend MesosCon for financial reasons. Equal access and diversity are important to MesosCon, and we aim to remove this obstacle for underrepresented attendee groups. We want you at MesosCon, and we want to help you get there!
Applications are beginning to be reviewed, and will be reviewed on a rolling basis. Apply today! We also encourage MesosCon attendees to donate toward the diversity scholarship program directly during the time of registration using the online system.</p>
<h2>Stay In Touch</h2>
<p>As we make future announcements about the conference, stay in the loop and help us get the word out! We’ll be tweeting from @ApacheMesos and @MesosCon, blogging on the <a href="http://mesos.apache.org/blog/">Apache Mesos blog</a>, and sharing updates on the project’s user@ mailing lists.</p>
<h2>About MesosCon</h2>
<p>MesosCon will take place August 20 - 21, 2015 at Sheraton Seattle in Seattle, WA with a hackathon taking place the day prior to the event. The standard attendee registration fee will be US$399 (Through July 31, 2015), and late registration US$499 beginning August 1, 2015. Students are eligible for a special registration fee of US$99 (Note that a valid student id will be required at onsite check-in).</p>
<p>Hope to see you at this year’s MesosCon!</p>
http://mesos.apache.org/blog/mesos-0-22-1-released/
Apache Mesos 0.22.1 Released
2015-05-12T00:00:00+00:00Adam B
<p>The latest Mesos release, 0.22.1 is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes fixes to containerizer recovery, sandbox permissions, and the Java and Python bindings. Also, certain fields of FrameworkInfo are now updated on reregistration.</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1795">MESOS-1795</a> - Assertion failure in state abstraction crashes JVM.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2161">MESOS-2161</a> - AbstractState JNI check fails for Marathon framework.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2461">MESOS-2461</a> - Slave should provide details on processes running in its cgroups</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2583">MESOS-2583</a> - Tasks getting stuck in staging.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2592">MESOS-2592</a> - The sandbox directory is not chown'ed if the fetcher doesn’t run.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2601">MESOS-2601</a> - Tasks are not removed after recovery from slave and mesos containerizer</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2614">MESOS-2614</a> - Update name, hostname, failover_timeout, and webui_url in master on framework re-registration</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2643">MESOS-2643</a> - Python scheduler driver disables implicit acknowledgments by default.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2668">MESOS-2668</a> - Slave fails to recover when there are still processes left in its cgroup</li>
</ul>
<p>Full release notes are available in the release <a href="https://github.com/apache/mesos/blob/master/CHANGELOG">CHANGELOG</a>.</p>
<p>Upgrading to 0.22.1 can be done seamlessly on a 0.22.0 cluster. If upgrading from an earlier version, please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrades</a> documentation.</p>
<h2>Contributors</h2>
<p>Thanks to all the contributors for 0.22.1: Benjamin Hindman, Benjamin Mahler, Ian Downes, Jie Yu, Joris Van Remoortere, Niklas Nielsen, and Timothy Chen.</p>
http://mesos.apache.org/blog/mesos-0-22-0-released/
Apache Mesos 0.22.0 Released
2015-03-25T00:00:00+00:00Niklas Q. Nielsen
<p>The latest Mesos release, 0.22.0, is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes several major features:</p>
<ul>
<li>Disk quota isolation in Mesos containerizer; refer to the <a href="http://mesos.apache.org/documentation/latest/mesos-containerizer/">containerization documentation</a> to enable disk quota monitoring and enforcement.</li>
<li>Support for explicitly sending status updates acknowledgements from schedulers; refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrades document</a> for upgrading schedulers.</li>
<li>Rate-limiting slave removal, to safeguard against unforeseen bugs which may lead to widespread slave removal.</li>
<li>Support for module hooks in task launch sequence. Refer to the <a href="http://mesos.apache.org/documentation/latest/modules/">modules documentation</a> for more information.</li>
<li>Anonymous modules: a new kind of module that does not receive any callbacks but coexists with its parent process.</li>
<li>New service discovery info in task info allows framework users to specify discoverability of tasks for external service discovery systems.
Refer to the <a href="http://mesos.apache.org/documentation/latest/app-framework-development-guide/">framework development guide</a> for more information.</li>
</ul>
<p>Furthermore, many bugs have been fixed in this release. For full release notes with all features and bug fixes, please refer to the <a href="https://github.com/apache/mesos/blob/master/CHANGELOG">CHANGELOG</a>.</p>
<p>Rolling upgrades may be done seamlessly, however some things have changed in 0.22.0 with regard to external tooling and upgrade paths; please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade guide</a> for detailed information on upgrading to this release from a 0.21.1 cluster.</p>
<h2>Contributors</h2>
<p>Thanks to the 35 contributors who made 0.22.0 possible:
Adam B, Alexander Rojas, Alexander Rukletsov, Ankur Chauhan, Benjamin Hindman, Benjamin Mahler, Bernd Mathiske, Chengwei Yang, Chi Zhang, Christos Kozyrakis, Cody Maloney, Connor Doyle, Dario Rexin, Dave Lester, David Robinson, Dominic Hamon, Evelina Dumitrescu, Gabriel Monroy, Ian Downes, Jake Farrell, Jiang Yan Xu, Jie Yu, Joerg Schad, Joris Van Remoortere, Kapil Arya, Kiyonari Harigae, Michael Park, Niklas Nielsen, Palak Choudhary, R.B. Boyer, Ryan Thomas, Samuel, Till Toenshoff, Timothy Chen and Vinod Kone.</p>
http://mesos.apache.org/blog/mesos-0-21-1-released/
Apache Mesos 0.21.1 Released
2015-01-20T00:00:00+00:00Tim Chen
<p>The latest Mesos release, 0.21.1 is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release includes bug fixes that fixes a few performance issues and isolator cleanup, and also includes a few improvements around building mesos on OSX and the docker containerizer.</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2047">MESOS-2047</a> - Isolator cleanup failures shouldn’t cause TASK_LOST.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2071">MESOS-2071</a> - Libprocess generates invalid HTTP</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2147">MESOS-2147</a> - Large number of connections slows statistics.json responses.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2182">MESOS-2182</a> - Performance issue in libprocess SocketManager.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1925">MESOS-1925</a> - Docker kill does not allow containers to exit gracefully</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-2113">MESOS-2113</a> - Improve configure to find apr and svn libraries/headers in OSX</li>
</ul>
<p>Full release notes are available in the release <a href="https://github.com/apache/mesos/blob/master/CHANGELOG">CHANGELOG</a>.</p>
<p>Upgrading to 0.21.1 can be done seamlessly on a 0.21.0 cluster. If upgrading from an earlier version, please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrades</a> documentation.</p>
<h2>Contributors</h2>
<p>Thanks to all the contributors for 0.21.1: Timothy Chen, Benjamin Mahler, Kapil Arya, Ryan Thomas, Jie Yu, Dario Rexin and Michael Park.</p>
http://mesos.apache.org/blog/mesoscon-2015-earlybird-registration-now-open/
MesosCon Early-Bird Registration Now Open
2015-01-14T00:00:00+00:00Dave Lester
<p>A limited number of early-bird tickets are now available for <a href="http://mesoscon.org">#MesosCon 2015</a>, an annual conference organized by the Apache Mesos community. MesosCon brings together users and developers to share and learn about the project and its growing ecosystem, and will take place in Seattle, WA.</p>
<blockquote class="twitter-tweet" lang="en"><p>.<a href="https://twitter.com/mesoscon">@MesosCon</a> early-bird registration is now open to the first 140 attendees! Join us! <a href="http://t.co/LcIclfttgq">http://t.co/LcIclfttgq</a> More info: <a href="http://t.co/Dnd5XWhoOH">http://t.co/Dnd5XWhoOH</a></p>— Apache Mesos (@ApacheMesos) <a href="https://twitter.com/ApacheMesos/status/555445970595368960">January 14, 2015</a></blockquote>
<p><a href="http://events.linuxfoundation.org/events/mesoscon/attend/register">Early-bird registration is open today</a> to the first 140 attendees / through February 14th, 2015.</p>
<p>MesosCon will take place August 20 - 21, 2015 at Sheraton Seattle in Seattle, WA. The cost of early-bird registration is US$299. The standard attendee registration fee will be US$399 (Through July 31, 2015), and late registration US$499 beginning August 1, 2015. Students are eligible for a special registration fee of US$99 (Note that a valid student id will be required at onsite check-in).</p>
<p>As part of the registration process, you’ll have an opportunity to donate to a <a href="http://events.linuxfoundation.org/events/mesoscon/attend/scholarship">Diversity Scholarship</a> which has been established for the conference. MesosCon’s scholarship program provides support to women, people of color, and people with disabilities who may not otherwise have the opportunity to attend MesosCon for financial reasons. Equal access and diversity are important to MesosCon, and we aim to remove this obstacle for underrepresented attendee groups. We encourage you to donate what you can! If you have any questions regarding the diversity scholarship, please contact <a href="mailto:mesoscon-diversity@googlegroups.com">mesoscon-diversity@googlegroups.com</a>.</p>
<p>On behalf of the #MesosCon Program Committee, we couldn’t be more excited about this year’s conference and what this means for the open source project’s community. We hope that you’ll join us!</p>
http://mesos.apache.org/blog/mesos-0-21-0-released/
Mesos 0.21.0 Released
2014-11-17T00:00:00+00:00Ian Downes
<p>The latest Mesos release, 0.21.0, is now available for <a href="http://mesos.apache.org/downloads/">download</a>. This version includes the following features and improvements:</p>
<ul>
<li>Task reconciliation for frameworks (<a href="https://issues.apache.org/jira/browse/MESOS-1407">MESOS-1407</a>)</li>
<li>Support for Mesos modules (<a href="https://issues.apache.org/jira/browse/MESOS-1931">MESOS-1931</a>)</li>
<li>Task status now includes source and reason (<a href="https://issues.apache.org/jira/browse/MESOS-343">MESOS-343</a>, <a href="https://issues.apache.org/jira/browse/MESOS-1143">MESOS-1143</a>)</li>
<li>A shared filesystem isolator (<a href="https://issues.apache.org/jira/browse/MESOS-1586">MESOS-1586</a>)</li>
<li>A pid namespace isolator (<a href="https://issues.apache.org/jira/browse/MESOS-1765">MESOS-1765</a>)</li>
</ul>
<p>Full release notes are available in the release <a href="https://github.com/apache/mesos/blob/master/CHANGELOG">CHANGELOG</a>.</p>
<h3>State Reconciliation</h3>
<p>Frameworks are now able to reconcile task state through the mesos API, to ensure that they remain eventually consistent in the face of failures. Read more about reconciliation <a href="http://mesos.apache.org/documentation/latest/reconciliation/">here</a>.</p>
<h3>Mesos Modules</h3>
<p>Experimental support for Mesos modules was introduced in Mesos 0.21.0;
please note the <a href="http://mesos.apache.org/documentation/latest/modules">disclaimer</a>.</p>
<p>Mesos modules provide a way to easily extend inner workings of Mesos by creating and using shared libraries that are loaded on demand. Modules can be used to customize Mesos without having to recompiling/relinking for each specific use case. Modules can isolate external dependencies into separate libraries, thus resulting into a smaller Mesos core. Modules also make it easy to experiment with new features. For example, imagine loadable allocators that contain a VM (Lua, Python, …) which makes it possible to try out new allocator algorithms written in scripting languages without forcing those dependencies into the project. Finally, modules provide an easy way for third parties to easily extend Mesos without having to know all the internal details.</p>
<p>For more details, please read the <a href="http://mesos.apache.org/documentation/latest/modules">modules documentation</a></p>
<h3>Task Status Includes Source and Reason</h3>
<p>The <code>TaskStatus</code> associated with a status update has relied on a single enum and free-form string to communicate the status. There are benefits to adding more data to the status update, and in Mesos 0.21.0, we have introduced the notion of a <code>Source</code> and <code>Reason</code>.</p>
<p>The source of a status update is, broadly speaking, the actor that changed the status. This can currently be Master, Slave, or Executor. Similarly, we have added an enumeration of reasons for the status update. The list can be found in <code>include/mesos/mesos.proto</code> and is fairly extensive.</p>
<p>Our hope is that frameworks can use the source and reason to better communicate status updates to end users, and make better scheduling decisions.</p>
<p>We have also introduced the notion of a <code>TASK_ERROR</code> state, distinct from <code>TASK_LOST</code>. The semantic difference is that tasks that are updated as lost can be rescheduled and should succeed, while tasks with status error will continue to fail if they are rescheduled. In Mesos 0.21.0 we do not send <code>TASK_ERROR</code> but it has been defined so frameworks can prepare to receive it. We will start sending it in Mesos 0.22.0.</p>
<h3>Shared Filesystem Isolator</h3>
<p>Mesos 0.21.0 introduces an optional shared filesystem isolator for the Mesos Containerizer. The isolator does not provide full filesystem encapsulation like Docker. It is intended for deployments where containers share the host’s root filesystem but certain parts of the filesystem should be made private to each container.</p>
<p>One example use-case is a private <code>/tmp</code> directory for each container. Processes running in the container will see a normal <code>/tmp</code> directory but it actually refers to a subdirectory of the executor work directory, e.g., the relative path <code>./tmp</code>. Processes will not be able to see files in the host’s <code>/tmp</code> directory or the <code>/tmp</code> directory of any other container.</p>
<p>This isolator is available only on Linux.</p>
<h3>Pid Namespace Isolator</h3>
<p>Mesos 0.21.0 also introduces an optional pid namespace isolator for the Mesos Containerizer. The isolator runs each container in its own pid namespace such that processes inside a container will not have visibility to processes running on the host or in other containers.</p>
<p>The Mesos Containerizer Launcher has been updated to use the pid namespace to terminate all processes when destroying a container. This avoids known kernel bugs and race conditions when using the freezer cgroup. If the container is not running inside a pid namespace (started with an earlier slave version), the launcher will fall back to the prior behavior and use the freezer cgroup.</p>
<p>This isolator is only available on Linux.</p>
<h3>Getting Involved</h3>
<p>We encourage you to try out this release and let us know what you think. If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks</h3>
<p>Thanks to the 27 <a href="https://github.com/apache/mesos/graphs/contributors">contributors</a> who made 0.21.0 possible:</p>
<p>Adam Bordelon, Alexander Rukletsov, Benjamin Hindman, Benjamin Mahler, Bernd Mathiske, Chengwei Yang, Chi Zhang, Cody Maloney, Dave Lester, Dominic Hamon, Evelina Dumitrescu, Ian Downes, Jie Yu, Joe Gordon, Joris Van Remoortere, Dharmesh Kakadia, Kapil Arya, Michael Park, Niklas Q. Nielsen, R.B. Boyer, Thomas Rampelberg, Till Toenshoff, Timothy Chen, Timothy St. Clair, Tobi Knaup, Vinod Kone.</p>
http://mesos.apache.org/blog/mesos-0-20-1-released/
Apache Mesos 0.20.1 Released
2014-09-23T00:00:00+00:00Adam B
<p>The latest Mesos release, 0.20.1 is now available for <a href="http://mesos.apache.org/downloads">download</a>. One of the major features of Mesos 0.20.0 was native Docker support, and that support has been improved in 0.20.1 by the following bug fixes and improvements:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1621">MESOS-1621</a> - Docker run networking should be configurable and support bridge network</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1724">MESOS-1724</a> - Can’t include port in DockerInfo’s image</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1730">MESOS-1730</a> - Should be an error if commandinfo shell=true when using docker containerizer</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1732">MESOS-1732</a> - Mesos containerizer doesn’t reject tasks with container info set</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1737">MESOS-1737</a> - Isolation=external result in core dump on 0.20.0</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1755">MESOS-1755</a> - Add docker support to mesos-execute</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1758">MESOS-1758</a> - Freezer failure leads to lost task during container destruction.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1762">MESOS-1762</a> - Avoid docker pull on each container run</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1770">MESOS-1770</a> - Docker with command shell=true should override entrypoint</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1809">MESOS-1809</a> - Modify docker pull to use docker inspect after a successful pull</li>
</ul>
<p>Full release notes are available in the release <a href="https://github.com/apache/mesos/blob/master/CHANGELOG">CHANGELOG</a>.</p>
<p>Upgrading to 0.20.1 can be done seamlessly on a 0.20.0 cluster. If upgrading from 0.19.x, please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrades</a> documentation.</p>
<h2>Contributors</h2>
<p>Thanks to all the contributors for 0.20.1: Timothy Chen, Jie Yu, Timothy St. Clair, Vinod Kone, Chris Heller, Kamil Domański, and Till Toenshoff.</p>
http://mesos.apache.org/blog/mesos-0-20-0-released/
Mesos 0.20.0 Released
2014-09-03T00:00:00+00:00Jie Yu
<p>The latest Mesos release, 0.20.0, is now available for <a href="http://mesos.apache.org/downloads/">download</a>. This version includes the following features and improvements:</p>
<ul>
<li>Native Docker support (<a href="https://issues.apache.org/jira/browse/MESOS-1524">MESOS-1524</a>)</li>
<li>Container level network monitoring (<a href="https://issues.apache.org/jira/browse/MESOS-1228">MESOS-1228</a>)</li>
<li>Framework authorization (<a href="https://issues.apache.org/jira/browse/MESOS-1342">MESOS-1342</a>)</li>
<li>Framework rate limiting (<a href="https://issues.apache.org/jira/browse/MESOS-1306">MESOS-1306</a>)</li>
<li>Building against installed third-party dependencies (<a href="https://issues.apache.org/jira/browse/MESOS-1071">MESOS-1071</a>)</li>
</ul>
<p>Full release notes are available in the release <a href="https://github.com/apache/mesos/blob/master/CHANGELOG">CHANGELOG</a>.</p>
<h3>Native Docker Support</h3>
<p>Mesos 0.20.0 introduces first-class <a href="https://www.docker.com/">Docker</a> support. Framework writers can now launch Docker containers for tasks by specifying <code>ContainerInfo</code> in <code>TaskInfo</code> or <code>ExecutorInfo</code>. The initial version allows the framework writers to specify Docker images for containers, and supports Docker primitives like volumes, entrypoints, etc.</p>
<p>Native Docker support is enabled by a new containerizer called DockerContainerizer, which can launch a Docker container as either a Task or an Executor. In 0.20.0, Mesos supports running multiple containerizers simultaneously and allows containerizers to be chosen dynamically based on task/executor info (<a href="https://issues.apache.org/jira/browse/MESOS-1527">MESOS-1527</a>). Therefore, users can enable native Docker support without changing the containerizer strategy for their existing tasks.</p>
<p>Details about native Docker support can be found in the <a href="http://mesos.apache.org/documentation/latest/docker-containerizer/">Docker containerizer documentation</a>.</p>
<h3>Container Level Network Monitoring</h3>
<p>Prior to Mesos 0.20.0, it was not possible to get the network statistics for each container, which made triaging or debugging network problems difficult in cases where multiple active tasks/executors were running on slave machines. Mesos 0.20.0 solves this problem by adding support for <a href="http://mesos.apache.org/documentation/latest/network-monitoring/">per-container network monitoring</a>. If you use the Mesos containerizer with the port_mapping isolator, network statistics for each active container can be retrieved through the <code>/monitor/statistics.json</code> endpoint on the slave.</p>
<p>This network monitoring solution is completely transparent to the tasks running on the slave. Tasks will not notice any difference to running on a slave without network monitoring enabled and were sharing the network of the slave. There is no need for users to change their service discovery mechanism.</p>
<h3>Framework Authorization</h3>
<p>Authorization support has been implemented via Access Control Lists (ACLs). Authorization allows:</p>
<ol>
<li>Frameworks to (re-)register with authorized ‘roles’</li>
<li>Frameworks to launch tasks/executors as authorized ‘users’</li>
<li>Authorized ‘principals’ to shutdown framework(s) through “/shutdown” HTTP endpoint</li>
</ol>
<p>For instance, operators can use authorization to only allow framework “foo” (and no other frameworks) to launch executors as “root”. For more details please read the <a href="http://mesos.apache.org/documentation/latest/authorization/">authorization documentation</a>.</p>
<h3>Framework Rate Limiting</h3>
<p>In a multi-framework environment, <a href="http://mesos.apache.org/documentation/latest/framework-rate-limiting/">framework rate limiting</a> aims to protect the throughput of high-SLA (e.g., production, service) frameworks by having the master throttle messages from other (e.g., development, batch) frameworks. Using this feature, an operator can specify the maximum number of queries per second and outstanding messages for the low-SLA frameworks on the master.</p>
<h3>Building Against Installed Third-party Dependencies</h3>
<p>Mesos 0.20.0 now supports building using system installed dependencies. Prior to this release, most of the third-party dependencies of Mesos were included in the project and statically linked into the resulting binaries and libraries. See <a href="https://issues.apache.org/jira/browse/MESOS-1071">MESOS-1071</a> for more details about these changes.</p>
<h3>Future Work</h3>
<p>Looking forward, here are a few upcoming features that are being worked on:</p>
<ul>
<li>Improvement of Docker support in Mesos (<a href="https://issues.apache.org/jira/browse/MESOS-1524">MESOS-1524</a>)</li>
<li>On top of network monitoring, evaluating and providing support for per-container network isolation (<a href="https://issues.apache.org/jira/browse/MESOS-1585">MESOS-1585</a>)</li>
<li>Continued work on state reconciliation for frameworks (<a href="https://issues.apache.org/jira/browse/MESOS-1407">MESOS-1407</a>)</li>
</ul>
<h3>Getting Involved</h3>
<p>We encourage you to try out this release and let us know what you think. If you run into any issues, please let us know on the <a href="https://mesos.apache.org/community">user mailing list and IRC</a>.</p>
<h3>Thanks</h3>
<p>Thanks to the 29 <a href="https://github.com/apache/mesos/graphs/contributors">contributors</a> who made 0.20.0 possible:</p>
<p>Adam B, Alexandra Sava, Anton Lindström, Benjamin Hindman, Benjamin Mahler, Bernardo Gomez Palacio, Bill Farner, Chengwei Yang, Chi Zhang, Connor Doyle, Craig Hansen-Sturm, Dave Lester, Dominic Hamon, Gaudio, Ian Downes, Isabel Jimenez, Jiang Yan Xu, Jie Yu, Ken Sipe, Niklas Nielsen, Thomas Rampelberg, Timothy Chen, Timothy St. Clair, Tobias Weingartner, Tom Arnfeld, Vinod Kone, Yifan Gu, Zuyu Zhang</p>
http://mesos.apache.org/blog/mesoscon-2014-hackathon-details-announced/
#MesosCon Hackathon Details Announced
2014-07-24T00:00:00+00:00Dave Lester
<p>After a <a href="http://mesoscon14.sched.org">full day of workshops and talks</a> at <a href="http://events.linuxfoundation.org/events/mesoscon">MesosCon</a>, we’re challenging participants to hack on <a href="http://mesos.apache.org">Mesos</a> during an <a href="http://www.atlassian.com">Atlassian</a>-sponsored hackathon taking place Friday, August 22nd. This event is an opportunity to contribute to the project, collaborate with other members of the community, and learn in the process.</p>
<p>We’ve identified two categories of hacks that we hope participants will focus on:</p>
<ul>
<li><em>Community Need</em>: These are issues logged in the <a href="https://issues.apache.org/jira/browse/MESOS/?selectedTab=com.atlassian.jira.jira-projects-plugin:issues-panel">Mesos issue tracker</a> that have received a large amount of activity (comments, votes, etc), and address some of the larger problems or pain points for the community. Suggested projects include new features, bugs, or documentation.</li>
<li><em>People’s Choice</em>: These contributions will be voted on by your peers, and our guidance is intentionally open-ended so teams will run wild with their ideas. We’re excited to see what you come up with.</li>
</ul>
<h2>Forming Groups</h2>
<p>We strongly encourage hackathon participants to use the public <a href="https://issues.apache.org/jira/browse/MESOS/">JIRA issue tracker</a> to log and discuss their ideas prior to the hackathon, using the <a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20MESOS%20AND%20labels%20%3D%20%22MesosConHack%22">MesosConHack label</a>. The <a href="http://mesos.apache.org/community/">project mailing lists and IRC</a> are good places to discover and share project ideas in advance.</p>
<p>Thursday evening of MesosCon, teams will form and ideas may be pitched. Additionally, project committers will be on-site to act as mentors and shepherds throughout the hackathon.</p>
<h2>Hackathon Rules</h2>
<ul>
<li>Actual development on submissions should begin the evening of August 21st and must not begin earlier. However, teams are encouraged to have community discussions prior to the event.</li>
<li>In order for an entry to be valid, a pull request or review board patch must be submitted.</li>
<li>Submitted applications must run.</li>
<li>Teams may have a maximum of four contributors.</li>
<li>A team may only win one category.</li>
<li>Submissions must be related to Mesos in some way.</li>
</ul>
<h2>Schedule</h2>
<h3>Thursday Evening</h3>
<p>Following sessions on day one of MesosCon, we will hold a hackathon kickoff that includes an explanation of the rules, and a chance for ideas to be pitched and teams formed. Hacking may begin at this time, although evening hacking is optional.</p>
<h3>Friday</h3>
<p>9:00 - Hacking begins at conference center</p>
<p>13:00 - Lunch arrives</p>
<p>17:00 - Coding stops, refreshments are served</p>
<p>17:15 - Re-explain the voting rules</p>
<p>17:20 - Presentations start - Each team gets 5 minutes to pitch what they built and show it off</p>
<p>18:00 - Voting occurs</p>
<p>18:15 - Winners announced</p>
<h2>Registration</h2>
<p>Interested in participating in the hackathon? Participants must <a href="https://www.regonline.com/register/login.aspx?eventID=1533330&MethodId=0&EventsessionId=">register for MesosCon</a>.</p>
<p>For new MesosCon registrants, you will see a checkbox at the time of registration. If you’ve already registered for MesosCon, you will be contacted to confirm whether you plan to participate in the hackathon.</p>
http://mesos.apache.org/blog/mesos-0-19-1-released/
Apache Mesos 0.19.1 Released
2014-07-21T00:00:00+00:00Ben Mahler
<p>The latest Mesos release, 0.19.1 is now available for <a href="http://mesos.apache.org/downloads">download</a>. This release fixes a bug in the Java and Python APIs in which the garbage collection of a MesosSchedulerDriver object leads to a framework unregistration.</p>
<p>More details can be found in <a href="https://issues.apache.org/jira/browse/MESOS-1550">MESOS-1550</a>, the <a href="http://mail-archives.apache.org/mod_mbox/mesos-user/201406.mbox/%3CCAFeOQnXVBBfgg4WQFatkHix2B1y-1o3EH3-mqtsgbEDeMH4Tbw%40mail.gmail.com%3E">report thread</a> from Whitney Sorenson, and the <a href="http://mail-archives.apache.org/mod_mbox/mesos-user/201406.mbox/%3CCAFeOQnVfhNo-mgkFsRn%3DvFsCQG%2Bj47xE3kdbFUDspWeOKjXA%3Dg%40mail.gmail.com%3E">follow up thread</a> from Benjamin Hindman.</p>
<p>This release has the following bug fixes:</p>
<ul>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1448">MESOS-1448</a> - Mesos Fetcher doesn’t support URLs that have 30X redirects.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1534">MESOS-1534</a> - Scheduler process is not explicitly terminated in the destructor of MesosSchedulerDriver.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1538">MESOS-1538</a> - A container destruction in the middle of a launch leads to CHECK failure.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1539">MESOS-1539</a> - No longer able to spin up Mesos master in local mode.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1550">MESOS-1550</a> - MesosSchedulerDriver should never, ever, call ‘stop’.</li>
<li><a href="https://issues.apache.org/jira/browse/MESOS-1551">MESOS-1551</a> - Master does not create work directory when missing.</li>
</ul>
<p>Upgrading to 0.19.1 can be done seamlessly on a 0.19.0 cluster. If upgrading from 0.18.x, please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrades</a> documentation.</p>
<h2>Contributors</h2>
<p>Thanks to all the contributors for 0.19.1: Tom Arnfeld, Ian Downes, Benjamin Hindman, Benjamin Mahler, Vinod Kone, Whitney Sorenson.</p>
http://mesos.apache.org/blog/mesos-community-survey-2014-results/
2014 Mesos Community Survey Results
2014-06-24T00:00:00+00:00Dave Lester
<p>Below are results of a community survey I led in May to get the pulse of the Apache Mesos community and improve our understanding of how others are using the software. We received a total of 55 responses.</p>
<p>While I think the responses below speak for themselves, I wanted to highlight a few things:</p>
<ul>
<li>A majority of responses came from adopters who have been using Mesos for more than 2 months.</li>
<li>Mesos had a <a href="http://www.netpromoter.com/why-net-promoter/know/">Net Promoter Score</a> of 41.5.</li>
<li>Documentation was the #1 community request for improving Mesos, followed by integrated Docker support. Unedited responses have been published as a <a href="https://gist.github.com/davelester/b40e3b5d0347540abf36">GitHub gist</a>.</li>
<li>36% of adopters are running Mesos in a cluster of 6-20 machines.</li>
<li>Almost half of Mesos adopters are running in private data centers. A majority of adopters running in the public cloud use AWS.</li>
<li>Hadoop, Marathon, and Chronos were the most-popular frameworks that are running in production.</li>
</ul>
<p>This is the first time we’ve run a community survey like this; feedback is welcome on how we may improve it when we run it again.</p>
<h3>How long have you been using Mesos?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_howlong.png" alt="How long have you been using Mesos" /></p>
<h3>How many machines are you running in your Mesos cluster?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_howmanymachines.png" alt="How many machines are you running in your Mesos cluster" /></p>
<h3>Frameworks: Are you using <a href="http://aurora.incubator.apache.org">Aurora</a> on Mesos?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_framework_aurora.png" alt="Are you using Aurora on Mesos" /></p>
<h3>Frameworks: Are you using <a href="https://github.com/mesosphere/cassandra-mesos">Cassandra</a> on Mesos?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_framework_cassandra.png" alt="Are you using Cassandra on Mesos" /></p>
<h3>Frameworks: Are you using <a href="https://github.com/airbnb/chronos">Chronos</a> on Mesos?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_framework_chronos.png" alt="Are you using Chronos on Mesos" /></p>
<h3>Frameworks: Are you using <a href="https://github.com/mesos/hadoop">Hadoop</a> on Mesos?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_framework_hadoop.png" alt="Are you using Hadoop on Mesos" /></p>
<h3>Frameworks: Are you using <a href="https://github.com/jenkinsci/mesos-plugin">Jenkins</a> on Mesos?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_framework_jenkins.png" alt="Are you using Jenkins on Mesos" /></p>
<h3>Frameworks: Are you using <a href="http://www.grandlogic.com/content/html_docs/products.shtml#jobserverprod">JobServer</a> on Mesos?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_framework_jobserver.png" alt="Are you using JobServer on Mesos" /></p>
<h3>Frameworks: Are you using <a href="https://github.com/mesosphere/marathon">Marathon</a> on Mesos?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_framework_marathon.png" alt="Are you using Marathon on Mesos" /></p>
<h3>Frameworks: Are you using <a href="http://spark.apache.org/">Spark</a> on Mesos?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_framework_spark.png" alt="Are you using Spark on Mesos" /></p>
<h3>Frameworks: Are you using <a href="https://github.com/mesosphere/storm-mesos">Storm</a> on Mesos?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_framework_storm.png" alt="Are you using Storm on Mesos" /></p>
<h3>Frameworks: Are you using a custom framework on Mesos?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_framework_custom.png" alt="Are you using a custom framework on Mesos" /></p>
<h3>Where do you run Mesos?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_wheredoyourun.png" alt="Where do you run Mesos" /></p>
<h3>How likely are you to recommend Mesos to your friends and colleagues?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_nps.png" alt="How likely are you to recommend Mesos to your friends and colleagues?" /></p>
<h3>Are you interested in contributing to Mesos in any of the following ways?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_contributions.png" alt="Are you interested in contributing to Mesos in any of the following way" /></p>
<h3>Is your company listed on our <a href="http://mesos.apache.org/documentation/latest/powered-by-mesos/">#PoweredByMesos</a> page? If not, can we list you?</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_poweredby.png" alt="Is your company listed on our #PoweredByMesos page?" /></p>
<p>(34 responses for “No, but please refrain from adding us at this time”, 8 responses for “No, please add us”, and 15 responses for “Yes, we’re on the list!”)</p>
<h3>Number of daily responses</h3>
<p><img src="/assets/img/blog/communitysurvey_2014_responses.png" alt="Number of daily response" /></p>
<h3>What features do you think are missing from Mesos?</h3>
<p>The unedited responses to this question have been published as a <a href="https://gist.github.com/davelester/b40e3b5d0347540abf36">GitHub gist</a>.</p>
http://mesos.apache.org/blog/mesos-0-19-0-released/
Apache Mesos 0.19.0 Released
2014-06-12T00:00:00+00:00Ben Mahler
<p>The latest Mesos release, 0.19.0 is now available for <a href="http://mesos.apache.org/downloads/">download</a>. This new version includes the following features and improvements:</p>
<ul>
<li>The master now persists the list of registered slaves in a durable replicated manner using the Registrar and the replicated log.</li>
<li>Alpha support for custom container technologies has been added with the <a href="https://github.com/apache/mesos/blob/0.19.0/src/slave/containerizer/external_containerizer.hpp#L74">ExternalContainerizer</a>.</li>
<li>Metrics reporting has been overhauled and is now exposed on <ip:port>/metrics/snapshot.</li>
<li>Slave Authentication: optionally, only authenticated slaves can register with the master.</li>
<li>Numerous bug fixes and stability improvements.</li>
</ul>
<p>Full release notes are available on <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311242&version=12326253">JIRA</a>.</p>
<h3>Registrar</h3>
<p>Mesos 0.19.0 introduces the <a href="https://cwiki.apache.org/confluence/display/MESOS/Registrar+Design+Document">“Registrar”</a>: the master now persists the list of registered slaves in a durable replicated manner. The previous lack of durable state was an intentional design decision that simplified failover and allowed masters to be run and migrated with ease. However, the stateless design had issues:</p>
<ul>
<li>In the event of a dual failure (slave fails while master is down), no lost task notifications are sent. This leads to a task running according to the framework but unknown to Mesos.</li>
<li>When a new master is elected, we may allow rogue slaves to reregister with the master. This leads to tasks running on the slave that are not known to the framework.</li>
</ul>
<p>Persisting the list of registered slaves allows failed over masters to detect slaves that do not reregister, and notify frameworks accordingly. It also allows us to prevent rogue slaves from reregistering; terminating the rogue tasks in the process.</p>
<p>The state is persisted using the <a href="http://mesos.apache.org/blog/mesos-0-17-0-released-featuring-autorecovery/">replicated log</a> (available since 0.9.0).</p>
<h3>External Containerization</h3>
<p>As <a href="http://mesos.apache.org/blog/mesos-0-18-0-released/">alluded to</a> during the containerization / isolation refactor in 0.18.0, the ExternalContainerizer has landed in this release. This provides <strong>alpha</strong> level support for custom containerization.</p>
<p><img src="/assets/img/documentation/containerizer_isolator_api.png" alt="Mesos Containerizer Isolator APIs" /></p>
<p>Developers can implement their own external containerizers to provide support for custom container technologies. Initial Docker support is now available through some community driven external containerizers: <a href="https://github.com/duedil-ltd/mesos-docker-containerizer">Docker Containerizer for Mesos</a> by Tom Arnfeld and <a href="https://github.com/mesosphere/deimos">Deimos</a> by Jason Dusek. Please reach out on the mailing lists with questions!</p>
<h3>Metrics</h3>
<p>Previously, Mesos components had to use custom metrics code and custom HTTP endpoints for exposing metrics. This made it difficult to expose additional system metrics and often required having an endpoint for each libprocess Process (Actor) for which metrics were desired. Having metrics spread across endpoints was operationally complex.</p>
<p>We needed a consistent, simple, and global way to expose metrics, which led to the creation of a metrics library within <a href="https://github.com/apache/mesos/tree/0.19.0/3rdparty/libprocess">libprocess</a>. All metrics are now exposed via /metrics/snapshot. The /stats.json endpoint remains for backwards compatibility.</p>
<h3>Upgrading</h3>
<p>For backwards compatibility, the “Registrar” will be enabled in a phased manner. By default, the “Registrar” is write-only in 0.19.0 and will be read/write in 0.20.0.</p>
<p>If running in high-availability mode with ZooKeeper, operators must now specify the <code>--work_dir</code> for the master, along with the <code>--quorum</code> size of the ensemble of masters. This means adding or removing masters must be done carefully! The best practice is to only ever add or remove a single master at a time and to allow a small amount of time for the replicated log to catch up on the new master. Maintenance documentation will be added to reflect this.</p>
<p>Please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrades</a> document, which details how to perform an upgrade from 0.18.x.</p>
<h3>Future Work</h3>
<p>Thanks to the Registrar, reconciliation primitives can now be provided to ensure that the state of tasks between Mesos and frameworks is kept consistent. This will remove the need for frameworks to implement out-of-band task reconciliation to inspect the state of slaves. Reconciliation work is being tracked at <a href="https://issues.apache.org/jira/browse/MESOS-1407">MESOS-1407</a>.</p>
<p>The addition of state through the Registrar opens up a rich set of possible features that were previously not possible due to the lack of persistent state in the master. These include:</p>
<ul>
<li>Cluster maintenance primitives (<a href="https://issues.apache.org/jira/browse/MESOS-1474">MESOS-1474</a>)</li>
<li>Repair automation (<a href="https://issues.apache.org/jira/browse/MESOS-695">MESOS-695</a>)</li>
<li>Global resource reservations</li>
</ul>
<h3>Getting Involved</h3>
<p>We encourage you to try out this release, and let us know what you think and if you hit any issues on the user mailing list. You can also get in touch with us via <a href="http://twitter.com/ApacheMesos">@ApacheMesos</a> or via <a href="http://mesos.apache.org/community/">mailing lists and IRC</a>.</p>
<h3>Thanks</h3>
<p>Thanks to the 32 contributors who made 0.19.0 possible:</p>
<p>Ashutosh Jain, Adam B, Alexandra Sava, Anton Lindström, Archana kumari, Benjamin Hindman, Benjamin Mahler, Bernardo Gomez Palacio, Bernd Mathiske, Charlie Carson, Chengwei Yang, Chi Zhang, Dave Lester, Dominic Hamon, Ian Downes, Isabel Jimenez, Jake Farrell, Jameel, Al-Aziz, Jiang Yan Xu, Jie Yu, Nikita Vetoshkin, Niklas Q. Nielsen, Ritwik Yadav, Sam Taha, Steven Phung, Till Toenshoff, Timothy St. Clair, Tobi Knaup, Tom Arnfeld, Tom Galloway, Vinod Kone, Vinson Lee</p>
http://mesos.apache.org/blog/mesoscon-2014-program-announced/
#MesosCon Program Announced, Now Including Hackathon
2014-06-11T00:00:00+00:00Dave Lester
<p>Today we’re excited to announce the <a href="http://mesoscon14.sched.org/">program</a> for <a href="http://events.linuxfoundation.org/events/mesoscon">#MesosCon</a> 2014, the first ever conference dedicated to the Mesos ecosystem. It will take place in Chicago August 21-22, 2014, and be co-located with <a href="http://events.linuxfoundation.org/events/linuxcon-north-america">LinuxCon</a> 2014.</p>
<p>MesosCon welcomes attendees who are advanced users, along with those that are new to the project. There will be time to socialize, share and learn from adopters to core committers. The first day will begin with an introductory <em>Mesos 101</em> workshop and keynote presentations from Mesos PMC lead Benjamin Hindman (Twitter) and John Wilkes (Google). Day one will include single track talks featuring:</p>
<ul>
<li>Mesos <a href="http://mesos.apache.org/documentation/latest/mesos-frameworks/">frameworks</a>: including <a href="http://aurora.incubator.apache.org">Apache Aurora</a>, <a href="https://github.com/mesosphere/marathon">Marathon</a>, <a href="https://github.com/apache/spark">Spark</a>, and a stream processing framework developed by Netflix.</li>
<li>An operations perspective on Mesos: including talks on the challenges of running an elastic cluster, and using <a href="http://www.docker.com">Docker</a> with Mesos.</li>
<li>Presentations from companies who use Mesos in production: including eBay, HubSpot, and Airbnb. These presenters will also participate in a panel discussion.</li>
</ul>
<p>The final day will include a Mesos hackathon, generously sponsored by Atlassian. In advance of the hackathon, the community will seed ideas for specific features and projects to work on the <a href="http://www.mail-archive.com/user@mesos.apache.org/">community mailing list</a>.</p>
<p><a href="https://www.regonline.com/Register/Checkin.aspx?EventID=1533330">Registration is open</a> and filling up quickly so register today. MesosCon is sponsored by Atlassian, eBay, Mesosphere, and Twitter. Interested companies can contact us regarding <a href="http://events.linuxfoundation.org/events/mesoscon/sponsor/become-a-sponsor">becoming a sponsor</a>.</p>
<p>For the latest updates on #MesosCon, we invite you to follow <a href="http://twitter.com/mesoscon">@MesosCon</a> on Twitter.</p>
http://mesos.apache.org/blog/mesos-0-18-1-and-0-18-2-released/
Mesos 0.18.1 and 0.18.2 Released
2014-05-29T00:00:00+00:00Niklas Quarfot Nielsen
<p>We recently tagged two new bug fixes including our latest release: version <a href="http://mesos.apache.org/downloads/">0.18.2</a>. A bug was introduced in 0.18.1, prompting the release of 0.18.2. If you’re using 0.18.1, it is highly recommended that you upgrade.</p>
<p>The latest version of Mesos is available on our <a href="http://mesos.apache.org/downloads/">downloads</a> page.</p>
<h3>Changes since 0.18.0</h3>
<p>With the release of 0.18.1, Maven is now used to compile and package Mesos' Java files. In addition, the Mesos Java artifacts shade the protocol buffer dependency, allowing frameworks to independently declare their protobuf dependency version. Furthermore, a new extract flag has been added to the URI protocol buffer which makes it possible to control whether the fetcher should extract a given artifact. Various bugs in the Mesos master, the containerizer and the JNI code have been fixed as well. (see <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311242&version=12326752">0.18.1 release notes</a>)</p>
<p>0.18.2 fixed a bug which caused the mesos-fetcher to not apply the executer bit. For additional details, please see: <a href="https://issues.apache.org/jira/browse/MESOS-1313">MESOS-1313</a>.</p>
<h3>Upgrading</h3>
<p>If upgrading from 0.18.0 or 0.18.1, an upgrade can be applied seamlessly to running clusters. However, Maven is now a build dependency and must be installed in order to build Mesos with Java support.</p>
<p>If upgrading from earlier versions than 0.18.0, please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">Upgrades</a> document.</p>
<h3>Getting Involved</h3>
<p>We encourage you to try out this release, and let us know what you think on the <a href="mailto:user@mesos.apache.org">user mailing list</a>. You can also get in touch with us via <a href="https://twitter.com/intent/user?screen_name=ApacheMesos">@ApacheMesos</a> or via <a href="https://mesos.apache.org/community">mailing lists and IRC</a>.</p>
http://mesos.apache.org/blog/mesos-0-18-0-released/
Mesos 0.18.0 Released
2014-04-10T00:00:00+00:00Ian Downes
<p>The latest Mesos release, <a href="http://mesos.apache.org/downloads/">0.18.0</a> is now available for <a href="http://mesos.apache.org/downloads">download</a>. A major part of the 0.18.0 release is a refactor of the project’s isolation strategy, making it easier to develop and use different isolation technologies with Mesos including alternative implementations. The refactor does not introduce any new isolation features; instead, we’ve established a set of APIs that others can use to extend Mesos.</p>
<h2>Isolation -> Containerization</h2>
<p>The first change is in terminology. The Mesos Slave now uses a Containerizer to provide an environment for each executor and its tasks to run in. Containerization includes resource isolation but is a more general concept that can encompass such things as packaging.</p>
<h2>Containerizer API</h2>
<p>The Containerizer API specifies the interface between the slave and a Containerizer. An internal containerization implementation, called MesosContainerizer, includes all isolation features present in earlier releases of Mesos: basic isolation on Posix systems and CPU and memory isolation using Cgroups on Linux systems.</p>
<p>One of the goals of the refactor was to enable alternative containerizer implementations. In particular, we encourage the community to add containerizer implementations which <em>delegate</em> containerization to other, existing technologies, e.g., LXC, Docker, VMware, Virtualbox, KVM and others.</p>
<p>The folks at <a href="http://mesosphere.io">Mesosphere</a> are working on one approach for a general ExternalContainerizer which provides a simple shim interface between the Mesos slave and an external program that performs isolation. When complete, the ExternalContainerizer could connect to any external containerizer such as, for example, <a href="https://github.com/mesosphere/deimos">Deimos</a>, another Mesosphere project, to use Docker for containerization.</p>
<h2>Isolator API</h2>
<p>A second goal of the refactor was to make it easier to both develop and use different isolation components of the MesosContainerizer. To do this, we separated the coordination logic from the isolation logic. Users of Mesos now have more granular control over which isolation components a slave uses.</p>
<p>MesosContainerizer accepts a list of Isolators, each implementing the Isolator API, to isolate resources. For example, the CgroupsMemIsolator uses the memory cgroup subsystem on Linux to limit the amount of memory available to an executor. We have plans to release new Isolators including a NetworkIsolator, a FilesystemIsolator, and a DiskIsolator and also to support isolation features provided by Linux namespaces such as pid and user namespaces.</p>
<p><img src="/assets/img/documentation/containerizer_isolator_api.png" alt="Mesos Containerizer Isolator APIs" /></p>
<h2>Cgroup layout change</h2>
<p>While we were doing the refactor we took the opportunity to update the cgroup layout to current best practices, making it easier to run Mesos on recent Linux distributions and alongside other cgroup users such as LXC, Docker, and systemd. The expected layout has changed from a single mount point for all controllers to a separate mount point for each controller. This change requires reconfiguring the host system for every slave and we highly recommend a system reboot.</p>
<p>Further details on the upgrade procedure are in the 0.18.0 <a href="http://mesos.apache.org/documentation/latest/upgrades">upgrade document</a> and, as always, please ask questions on the <a href="http://mesos.apache.org/community">mailing list</a>, file tickets via <a href="https://issues.apache.org/jira/browse/MESOS">JIRA</a>, or discuss on <a href="http://mesos.apache.org/community">IRC</a>.</p>
http://mesos.apache.org/blog/mesos-community-update-1/
Mesos Community Update #1
2014-03-28T00:00:00+00:00Matt Trifiro
<p><em>This is a cross post from the <a href="http://mesosphere.io/community/2014/03/20/mesos-community-update-1/">Mesosphere blog</a>.</em></p>
<p>Our community is in the midst of a revolution. Mesos is no longer an edge technology, only for the Twitters and Airbnbs of the world; it’s going mainstream and we are at its center. There is more activity on the Apache mailing lists, more commits in the code, more frameworks being developed, more developers building tools around Mesos—and, perhaps most excitingly, there are more companies deploying Mesos into production.</p>
<p>All of this activity makes it harder to keep up!</p>
<p>The Mesosphere team wants to help you keep up with all of the community activity by curating a concise semi-regular update, of which this is the first. We are looking to share the most interesting and important activities in the Mesos community.</p>
<h3>Here are some recent highlights:</h3>
<ul>
<li><p>Venture capital icon <a href="http://en.wikipedia.org/wiki/Vinod_Khosla">Vinod Khosla</a> gave a call-out to Mesos in his 2014 Open Networking Summit keynote, where he waxed about the need for a data center OS. The Mesos portion begins at 24:07 in the <a href="http://youtu.be/q61VkqZRjck?t=24m7s%20">ONS2014 Keynote YouTube video</a>.</p></li>
<li><p>Lab49 software engineer <a href="https://twitter.com/anvarkarimson">Anvar Karimson</a> re-implemented Stripe’s legendary Capture the Flag (CTF) system on Mesos. Read his blow-by-blow description here: <a href="http://karimson.com/posts/ctf-mesos/">Running Stripe CTF 2.0 on Mesos</a>.</p></li>
<li><p>Community member <a href="https://twitter.com/barton_tomas">Tomas Barton</a> gave an introductory talk on Mesos at the <a href="http://www.installfest.cz/if14/">InstallFest</a> in Prague. In his 59-slide talk, he covered everything from workload balancing to Mesos fault tolerance. View his presentation on SlideShare: <a href="http://www.slideshare.net/tomasbart/introduction-to-apache-mesos">Introduction to Apache Mesos</a>.</p></li>
<li><p>On February 27, <a href="http://mesos.apache.org/blog/mesos-0-17-0-released-featuring-autorecovery/">Mesos 0.17.0 was released</a> and you can <a href="http://mesosphere.io/downloads/#apache-mesos-0.17.0">download it here</a>. The 0.17.0 release features auto-recovery of the replicated log, which enhances Mesos’s high-availability and fault-tolerance. Read more here: <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311242&version=12325669">Mesos 0.17.0 release notes</a></p></li>
<li><p>On the path toward fully implementing the <a href="https://cwiki.apache.org/confluence/display/MESOS/Registrar+Design+Document">Mesos Registrar Design</a>, we saw progress toward creating <a href="https://issues.apache.org/jira/browse/MESOS-764">persistence of state information for slaves</a>. Placing a small amount of state information in highly-available storage will make recovery of slaves faster and more graceful (this has been <a href="https://issues.apache.org/jira/browse/MESOS-295">lacking for a while</a>).</p></li>
<li><p>Mesosphere has packaged a pre-release version of Mesos 0.18.0 (release candidate 4) that you can now <a href="http://mesosphere.io/downloads/#apache-mesos-0.18.0-rc4">download</a>. The primary features of the <a href="https://github.com/apache/mesos/blob/0.18.0-rc4/CHANGELOG">upcoming 0.18.0 release</a> are changes that make it easier to insert pluggable container technologies, like Docker. 0.18.0 foreshadows some pretty interesting Docker integrations with Mesos.</p></li>
</ul>
<h3>Upcoming events:</h3>
<ul>
<li><p>BBQ at Mesosphere HQ, 6-8 pm, March 28th. Come join us for a low-key BBQ. 145A Hampshire St, San Francisco (near Potrero @ 15th). Swing by and meet <a href="http://mesosphere.io/team/">our team</a>.</p></li>
<li><p><a href="http://apacheconnorthamerica2014.sched.org/">ApacheCon</a>, April 7-9 in Denver. Come see two Mesos presentations: <a href="http://apacheconnorthamerica2014.sched.org/event/d83ffc7d7c56620474eac1a2d8f09967">Mesos: Elastically Scalable Operations, Simplified</a> and <a href="http://apacheconnorthamerica2014.sched.org/event/803cb2a6f321ee02957b1c4eb4ebc01c">Building and Running Distributed Systems using Apache Mesos</a>.</p></li>
</ul>
<p>We hope to publish these updates at least a few times each month, and we’re interested in your suggestions. Please let us know what you think about the length, content and style, and send us your tips on what to publish each week. Do you have something to share? Write to us at <a href="mailto:community@mesosphere.io">community@mesosphere.io</a>.</p>
<p><em>This post was co-authored by Abhishek Parolkar. Follow him on Twitter at <a href="https://twitter.com/parolkar">@parolkar</a>.</em></p>
http://mesos.apache.org/blog/mesos-0-17-0-released-featuring-autorecovery/
Mesos 0.17.0 Released, Featuring Autorecovery for the Replicated Log
2014-03-13T00:00:00+00:00Jie Yu
<p>The latest Mesos release, <a href="http://mesos.apache.org/downloads/">0.17.0</a> was made available for <a href="http://mesos.apache.org/downloads">download</a> last week, introducing a new autorecovery feature for the replicated log that handles additional disk failure and operator error scenarios.</p>
<h2>What is the replicated log?</h2>
<p>The replicated log is a distributed <em>append-only</em> log that stores arbitrary data as entries. Replication comes into play as at least a quorum of log replicas are stored on different machines, protecting the data from individual disk or network failures. The replicated log uses the <a href="http://en.wikipedia.org/wiki/Paxos_(computer_science%29">Paxos algorithm</a>. We will describe additional implementation details in both an upcoming blog post and project documentation.</p>
<p>Mesos has had replicated log since version 0.9.0 and <a href="http://aurora.incubator.apache.org/">Apache Aurora</a>, a framework used in production at Twitter, uses replicated log for its persistent storage. Additionally, the Mesos master will start using the replicated log to persist some cluster state with the introduction of the <a href="https://issues.apache.org/jira/browse/MESOS-764">registrar</a>.</p>
<h2>New feature in Mesos 0.17.0: <em>autorecovery</em></h2>
<p>Before Mesos version 0.17.0, the replicated log could become inconsistent if a replica’s disk was replaced or the data was accidentally deleted by an operator. That meant that an operator couldn’t safely, or easily, replace a replica’s disk while the other replicas kept running. Instead, an operator had to stop all replicas, replace the disk, copy a pre-existing log, then restart all the replicas.</p>
<p>As of this release, a replica with a new disk can automatically recover without operator involvement. As long as a quorum of replicas are available, applications using the replicated log won’t notice a thing!</p>
<h2>Future work</h2>
<p>The new autorecovery feature enables replacing a single replica at a time, but it does not allow an operator to easily add or remove replicas, known as <em>reconfiguration</em>. You can follow <a href="https://issues.apache.org/jira/browse/MESOS-683">MESOS-683</a> for the latest on this future work.</p>
http://mesos.apache.org/blog/mesos-0-16-0-released/
Mesos 0.16.0 Released
2014-02-11T00:00:00+00:00Yan Xu
<p>We recently released Mesos v0.16.0 on our <a href="http://mesos.apache.org/downloads/">downloads</a> page. It includes major <a href="https://issues.apache.org/jira/browse/MESOS-496">refactoring work</a> of the leading master election and detection process. This improves the reliability and flexibility of running multiple masters in your cluster, which provides Mesos with high availability.</p>
<p>In high availability mode, if a leading master machine fails, Mesos holds elections to determine a new leader. Slave machines and schedulers detect the new leading master and connect to it, without disrupting services running on Mesos. Leader election implementation details, including how it works with <a href="http://zookeeper.apache.org">Zookeeper</a>, are detailed in the <a href="http://mesos.apache.org/documentation/latest/high-availability/">high availablity documentation</a>.</p>
<h2>What’s Changed</h2>
<p>Aside from the refactoring, v0.16.0 includes fixes for bugs which caused incorrect termination of Mesos masters and slaves:</p>
<ul>
<li>Fixed ZooKeeper related bugs which terminated Mesos processes instead of automatically retrying them: <a href="https://issues.apache.org/jira/browse/MESOS-463">MESOS-463</a>, <a href="https://issues.apache.org/jira/browse/MESOS-465">MESOS-465</a>, <a href="https://issues.apache.org/jira/browse/MESOS-814">MESOS-814</a>.</li>
<li>Non-leading Master now stays up after ZooKeeper session expiration or after it is partitioned from ZooKeeper.</li>
<li>Slave no longer attempts to recover checkpointed data after a reboot: <a href="https://issues.apache.org/jira/browse/MESOS-844">MESOS-844</a>.</li>
</ul>
<p>Click to read the full <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311242&version=12325295">release notes</a>.</p>
<h2>Upgrading</h2>
<p>To upgrade a live cluster, please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">Upgrades document</a>.</p>
<h2>Getting Involved</h2>
<p>We encourage you to try out this release, and let us know what you think on the <a href="mailto:user@mesos.apache.org">user mailing list</a>. You can also get in touch with us via <a href="https://twitter.com/intent/user?screen_name=ApacheMesos">@ApacheMesos</a> or via <a href="https://mesos.apache.org/community">mailing lists and IRC</a>.</p>
http://mesos.apache.org/blog/framework-authentication-in-apache-mesos-0-15-0/
Mesos 0.15 and Authentication Support
2014-01-09T00:00:00+00:00Vinod Kone
<p>With the latest Mesos release, <a href="http://mesos.apache.org/downloads/">0.15.0</a>, we are pleased to report that we’ve added Authentication support for frameworks (see <a href="https://issues.apache.org/jira/browse/MESOS-418">MESOS-418</a>) connecting to Mesos. In a nutshell, this feature allows only authenticated frameworks to register with Mesos and launch tasks. Authentication is important as it prevents rogue frameworks from causing problems that may impact the usage of resources within a Mesos cluster.</p>
<h2>How it works</h2>
<ul>
<li><p><strong>SASL</strong></p>
<p> Mesos uses the <a href="http://asg.web.cmu.edu/sasl/">Cyrus SASL library</a> to provide authentication. SASL is a very flexible authentication framework that allows two endpoints to authenticate with each other and also has support for various authentication mechanisms (ANONYMOUS, PLAIN, CRAM-MD5, GSSAPI etc).</p>
<p> In this release, Mesos uses SASL with the CRAM-MD5 authentication mechanism. The process for enabling authentication begins with the creation of an authentication credential that is unique to the framework. This credential constitutes a <strong>principal</strong> and <strong>secret</strong> pair, where <strong>principal</strong> refers to the identity of the framework. Note that the <strong>principal</strong> is different from the framework <strong>user</strong> (the Unix user which executors run as) or the resource <strong>role</strong> (role that has reserved certain resources on a slave). These credentials should be shipped to the Mesos master machines, as well as given access to the framework, meaning that both the framework and the Mesos masters should be started with these credentials.</p>
<p> Once authentication is enabled, Mesos masters only allow authenticated frameworks to register. Authentication for frameworks is performed under the hood by the new scheduler driver.</p></li>
</ul>
<p>For specific instructions on how to do this please read the <a href="http://mesos.apache.org/documentation/latest/upgrades/">upgrade</a> instructions.</p>
<h2>Looking ahead</h2>
<ul>
<li><p><strong>Adding authentication support for slaves</strong></p>
<p> Similar to adding authentication support to frameworks, it would be great to add authentication support to the slaves. Currently any node in the network can run a Mesos slave process and register with the Mesos master. Requiring slaves to authenticate with the master before registration would prevent rogue slaves from causing problems (e.g., DDoSing the master, getting access to users tasks etc) in the cluster.</p></li>
<li><p><strong>Integrating with Kerberos</strong></p>
<p> Currently the authentication support via shared secrets between frameworks and masters is basic to benefit usability. To improve upon this basic approach, a more powerful solution would be to integrate with an industry standard authentication service like <a href="http://en.wikipedia.org/wiki/Kerberos_(protocol">Kerberos</a>). A nice thing about SASL and one of the reasons we picked it is because of its support for integration with GSSAPI/Kerberos. We plan to leverage this support to integrate Kerberos with Mesos.</p></li>
<li><p><strong>Data encryption</strong></p>
<p> Authentication is only part of the puzzle when it comes to deploying and running applications securely in the cloud. Another crucial component is data encryption. Currently all the messages that flow through the Mesos cluster are un-encrypted making it possible for intruders to intercept and potentially control your task. We plan to add encryption support by adding SSL support to <a href="https://github.com/3rdparty/libprocess">libprocess</a>, the low-level communication library that Mesos uses which is responsible for all network communication between Mesos components.</p></li>
<li><p><strong>Authorization</strong></p>
<p> We are also investigating authorizing principals to allow them access to only a specific set of operations like launching tasks or using resources. In fact, you could imagine a world where an authenticated <strong>principal</strong> will be authorized to on behalf of a subset of <strong>user</strong>s and <strong>role</strong>s for launching tasks and accepting resources respectively. This authorization information could be stored in a directory service like LDAP.</p></li>
</ul>
<h3>Thanks</h3>
<p>While a lot of people contributed to this feature, we would like to give special thanks to <a href="https://twitter.com/ilimugur">Ilim Igur</a>, our <a href="http://www.google-melange.com/gsoc/homepage/google/gsoc2013">Google Summer of Code</a> intern who started this project and contributed to the intial design and implementation.</p>
<p>If you are as excited as us about this feature please go ahead and play with <a href="http://mesos.apache.org">Mesos 0.15.0</a> and let us know what you think. You can get in touch with us via <a href="http://mesos.apache.org/community/">our mailing lists or IRC</a>.</p>
http://mesos.apache.org/blog/niklas-nielsen-becomes-mesos-committer/
Niklas Nielsen Becomes Newest Mesos Committer and PMC Member
2013-12-16T00:00:00+00:00Apache Mesos
<p>The Apache Mesos PMC has voted to make <a href="http://mesosphere.io/2013/10/24/niklas-nielsen-joins-mesosphere/">Niklas Nielsen</a> the project’s newest committer and PMC member. Niklas is an engineer at Engineer at <a href="http://mesosphere.io">Mesosphere</a>, and has a background in virtual machines, compilers and supercomputer tooling (distributed debuggers, profilers and so on).</p>
<p>Nielsen has particular interests in performance, scalability and tooling to give insights in otherwise fairly complex setups. Welcome, Niklas!</p>
http://mesos.apache.org/blog/mesos-0-14-2-released/
Apache Mesos 0.14.2 Released
2013-11-12T00:00:00+00:00Ben Mahler
<p>We recently released Mesos 0.14.2, a bugfix release with only a minor change related to cgroups isolation in 0.14.1. If you’re using 0.14.1 with cgroups isolation, it is recommended to upgrade to avoid unnecessary out-of-memory (OOM) killing of jobs. The latest version of Mesos is available on our <a href="http://mesos.apache.org/downloads/">downloads</a> page.</p>
<h3>Upgrading</h3>
<p>If upgrading from 0.14.x, this upgrade can be applied seamlessly to running clusters. However, if you’re upgrading from 0.13.x on a running cluster, please refer to the <a href="http://mesos.apache.org/documentation/latest/upgrades/">Upgrades</a> document, which details how a seamless upgrade from 0.13.x to 0.14.x can be performed.</p>
<h3>Changes since 0.14.1</h3>
<p>With the release of 0.14.1, when using cgroups isolation, the OOM semantics were altered to enable the kernel OOM killer and to use the memory soft limit combined with memory threshold notifications to induce OOMs in user-space. This was done to attempt to capture memory statistics at the time of OOM for diagnostic purposes. However, this proved to trigger unintended OOMs as the memory purging that occurs when the hard limit is reached was no longer occurring.</p>
<p>0.14.2 no longer uses threshold notifications; the memory hard limit is now used instead to preserve the previous OOM semantics.</p>
<p>For additional details, please see:
<a href="https://issues.apache.org/jira/browse/MESOS-755">MESOS-755</a>,
<a href="https://issues.apache.org/jira/browse/MESOS-762">MESOS-762</a>.</p>
http://mesos.apache.org/blog/slave-recovery-in-apache-mesos/
Slave Recovery in Apache Mesos
2013-10-23T00:00:00+00:00Vinod Kone
<p>High availability is one of the key features of Mesos. For example, a typical Mesos cluster in production involves 3-5 masters with one acting as <em>leader</em> and the rest on standby. When a leading master fails due to a crash or goes offline for an upgrade, a standby master automatically becomes the leader without causing any disruption to running services. Leader election is currently performed by using <a href="http://zookeeper.apache.org/">ZooKeeper</a>.</p>
<p>With the latest Mesos release, <a href="http://mesos.apache.org/downloads/">0.14.1</a>, we are bringing high availability to the slaves by introducing a new feature called <em>Slave Recovery</em>. In a nutshell, slave recovery enables:</p>
<ol>
<li><p>Executors/tasks to keep running when the slave process is down.</p></li>
<li><p>A restarted slave process to reconnect with running executors/tasks on the slave.</p></li>
</ol>
<h2>Why it matters</h2>
<p>A recoverable slave is critical for running services in production on Mesos for several reasons:</p>
<ul>
<li><p><strong>Stateful services</strong></p>
<p> In a typical production environment there are stateful services (e.g., caches) running in the cluster. It is not uncommon for these services to have a high startup time (e.g., cache warm up time of a few hours). Even in the analytics world, there are cases where a single task is responsible for doing work that takes hours to complete. In such cases a restart of the slave (e.g, crash or upgrade) will have a huge impact on the service. While sharding the service wider mitigates this impact it is not always possible to do so (e.g, legacy services, data locality, application semantics). Such stateful applications would benefit immensely from running under a more resilient slave.</p></li>
<li><p><strong>Faster cluster upgrades</strong></p>
<p> It is important for clusters to frequently upgrade their infrastructure to stay up-to-date with the latest features and bug fixes. A typical Mesos slave upgrade involves stopping the slave, upgrading the slave libraries and starting the slave. In production environments there is always tension between upgrading the infrastructure frequently and the need to not impact long running services as much as possible. With respect to Mesos upgrades, if upgrading the slave binary has no impact on the underlying services, then it is a win for both the cluster operators and the service owners.</p></li>
<li><p><strong>Resilience against slave failures</strong></p>
<p> While upgrading the slaves is most often the reason for restarting slaves, there might be other causes for a slave to fail. A slave crash could happen due to a bug in the slave code or due to external factors like a bug in the kernel or ZooKeeper client library. Typically such crashes are temporary and a restart of the slave is enough to correct the situation. If such slave restarts do not affect applications running on the slave it is a huge win for the applications.</p></li>
</ul>
<h2>How it works</h2>
<ul>
<li><p><strong>Checkpointing</strong></p>
<p> Slave recovery works by having the slave checkpoint enough information (e.g., task information, executor information, status updates) about the running tasks and executors to local disk. Once the slave and the framework(s) enable checkpointing, any subsequent slave restarts would recover the checkpointed information and reconnect with the executors. When a checkpointing slave process goes down, both the leading master and the executors running on the slave host wait for the slave to come back up and reconnect. A nice thing about slave recovery is that frameworks and their executors/tasks are oblivious to a slave restart.</p></li>
<li><p><strong>Executor Driver Caching</strong></p>
<p> As part of this feature, the executor driver has also been improved to make it more resilient in the face of a slave failure. As an example, status updates sent by the executor while the slave is down are cached by the driver and sent to the slave when it reconnects with the restarted slave. Since this is all handled by the executor driver, framework/executor writers do not have to worry about it! The executors can keep sending status updates for their tasks while remaining oblivious to the slave being up or down.</p></li>
<li><p><strong>Reliable status updates</strong></p>
<p> Another benefit of slave checkpointing the status updates is that now updates are more reliably delivered to frameworks in the face of failures. Before slave recovery if the slave fails at the same time that a master is failing over, no TASK_LOST updates for tasks running on the slave were sent to the framework. This is partly because the Mesos master is stateless. A failed over master reconstructs the cluster state from the information given to it by reregistering slaves and frameworks. With slave recovery, status updates and tasks are no longer lost when slaves fail. Rather, the slave recovers tasks, status updates and reconnects with the running executors. Even if an executor does terminate when the slave is down, a recovered slave knows about it from its checkpointed state and reliably sends TASK_LOST updates to the framework.</p></li>
</ul>
<p>For more information about how to enable slave recovery in your cluster, please refer to the <a href="https://github.com/apache/mesos/blob/master/docs/Slave-Recovery.md">documentation</a>.</p>
<h2>Looking ahead</h2>
<ul>
<li><p><strong>Easy Executor/Task Upgrades</strong></p>
<p> In a similar vein to how slave recovery makes upgrading a Mesos cluster easy, we would like to enable frameworks to upgrade their executors/tasks as well. Currently the only way to upgrade an executor/task is to kill the old executor/task and launch the new upgraded executor/task. For the same reasons as we have discussed earlier this is not ideal for stateful services. We are currently investigating proper primitives to provide to frameworks to do such upgrades, so that not every framework have to (re-)implement that logic.</p></li>
<li><p><strong>State Reconciliation</strong></p>
<p> While slave recovery greatly improves the reliability of delivering status updates there are still some rare cases where updates could be lost. For example if a slave crashes when a master is failing over and never comes back then the new leading master doesn’t know about the lost slave and executors/tasks running on it. In addition to status updates, any driver methods (e.g., launchTasks, killTask) invoked when the master is failing over are silently dropped. Currently, frameworks are responsible for reconciling their own task state using the <a href="https://github.com/apache/mesos/blob/master/include/mesos/scheduler.hpp#L290">reconciliation API</a>. We are currently investigating ways to provide even better guarantees around reconciliation.</p></li>
<li><p><strong>Self Updates of Mesos</strong></p>
<p> Currently, updating Mesos involves a cluster operator to manually upgrade the master and slave binaries and roll them in a specific manner (e.g., masters before slaves). But what if Mesos can update itself! It is not hard to imagine a future where Mesos masters can orchestrate the upgrade of slaves and maybe also upgrade one another! This would also help making rollbacks easy incase an upgrade doesn’t work because it would be much easier Mesos to check if various components are working as expected.</p></li>
</ul>
<p>So what are you waiting for? Go ahead and give <a href="http://mesos.apache.org">Mesos</a> a whirl and <a href="mailto:user@mesos.apache.org">let us know</a> what you think.</p>