Apache Mesos
Public Member Functions | Static Public Member Functions | List of all members
mesos::internal::master::allocator::MesosAllocator< AllocatorProcess > Class Template Reference

#include <allocator.hpp>

Inheritance diagram for mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >:
mesos::allocator::Allocator

Public Member Functions

 ~MesosAllocator ()
 
void initialize (const Duration &allocationInterval, const lambda::function< void(const FrameworkID &, const hashmap< std::string, hashmap< SlaveID, Resources >> &)> &offerCallback, const lambda::function< void(const FrameworkID &, const hashmap< SlaveID, UnavailableResources > &)> &inverseOfferCallback, const Option< std::set< std::string >> &fairnessExcludeResourceNames=None(), bool filterGpuResources=true, const Option< DomainInfo > &domain=None())
 Initializes the allocator when the master starts up. More...
 
void recover (const int expectedAgentCount, const hashmap< std::string, Quota > &quotas)
 Informs the allocator of the recovered state from the master. More...
 
void addFramework (const FrameworkID &frameworkId, const FrameworkInfo &frameworkInfo, const hashmap< SlaveID, Resources > &used, bool active, const std::set< std::string > &suppressedRoles)
 Adds a framework to the Mesos cluster. More...
 
void removeFramework (const FrameworkID &frameworkId)
 Removes a framework from the Mesos cluster. More...
 
void activateFramework (const FrameworkID &frameworkId)
 Activates a framework in the Mesos cluster. More...
 
void deactivateFramework (const FrameworkID &frameworkId)
 Deactivates a framework in the Mesos cluster. More...
 
void updateFramework (const FrameworkID &frameworkId, const FrameworkInfo &frameworkInfo, const std::set< std::string > &suppressedRoles)
 Updates capabilities of a framework in the Mesos cluster. More...
 
void addSlave (const SlaveID &slaveId, const SlaveInfo &slaveInfo, const std::vector< SlaveInfo::Capability > &capabilities, const Option< Unavailability > &unavailability, const Resources &total, const hashmap< FrameworkID, Resources > &used)
 Adds or re-adds an agent to the Mesos cluster. More...
 
void removeSlave (const SlaveID &slaveId)
 Removes an agent from the Mesos cluster. More...
 
void updateSlave (const SlaveID &slave, const SlaveInfo &slaveInfo, const Option< Resources > &total=None(), const Option< std::vector< SlaveInfo::Capability >> &capabilities=None())
 Updates an agent. More...
 
void addResourceProvider (const SlaveID &slave, const Resources &total, const hashmap< FrameworkID, Resources > &used)
 Add resources from a local resource provider to an agent. More...
 
void activateSlave (const SlaveID &slaveId)
 Activates an agent. More...
 
void deactivateSlave (const SlaveID &slaveId)
 Deactivates an agent. More...
 
void updateWhitelist (const Option< hashset< std::string >> &whitelist)
 Updates the list of trusted agents. More...
 
void requestResources (const FrameworkID &frameworkId, const std::vector< Request > &requests)
 Requests resources for a framework. More...
 
void updateAllocation (const FrameworkID &frameworkId, const SlaveID &slaveId, const Resources &offeredResources, const std::vector< ResourceConversion > &conversions)
 Updates allocation by applying offer operations. More...
 
process::Future< NothingupdateAvailable (const SlaveID &slaveId, const std::vector< Offer::Operation > &operations)
 Updates available resources on an agent based on a sequence of offer operations. More...
 
void updateUnavailability (const SlaveID &slaveId, const Option< Unavailability > &unavailability)
 Updates unavailability for an agent. More...
 
void updateInverseOffer (const SlaveID &slaveId, const FrameworkID &frameworkId, const Option< UnavailableResources > &unavailableResources, const Option< mesos::allocator::InverseOfferStatus > &status, const Option< Filters > &filters)
 
process::Future< hashmap< SlaveID, hashmap< FrameworkID, mesos::allocator::InverseOfferStatus > > > getInverseOfferStatuses ()
 Retrieves the status of all inverse offers maintained by the allocator. More...
 
void recoverResources (const FrameworkID &frameworkId, const SlaveID &slaveId, const Resources &resources, const Option< Filters > &filters)
 Recovers resources. More...
 
void suppressOffers (const FrameworkID &frameworkId, const std::set< std::string > &roles)
 Suppresses offers. More...
 
void reviveOffers (const FrameworkID &frameworkId, const std::set< std::string > &roles)
 Revives offers to this framework for the specified roles. More...
 
void setQuota (const std::string &role, const Quota &quota)
 Informs the allocator to set quota for the given role. More...
 
void removeQuota (const std::string &role)
 Informs the allocator to remove quota for the given role. More...
 
void updateWeights (const std::vector< WeightInfo > &weightInfos)
 Updates the weight associated with one or more roles. More...
 
- Public Member Functions inherited from mesos::allocator::Allocator
 Allocator ()
 
virtual ~Allocator ()
 
virtual void updateInverseOffer (const SlaveID &slaveId, const FrameworkID &frameworkId, const Option< UnavailableResources > &unavailableResources, const Option< InverseOfferStatus > &status, const Option< Filters > &filters=None())=0
 Updates inverse offer. More...
 

Static Public Member Functions

static Try< mesos::allocator::Allocator * > create ()
 
- Static Public Member Functions inherited from mesos::allocator::Allocator
static Try< Allocator * > create (const std::string &name)
 Attempts either to create a built-in DRF allocator or to load an allocator instance from a module using the given name. More...
 

Constructor & Destructor Documentation

template<typename AllocatorProcess >
mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::~MesosAllocator ( )

Member Function Documentation

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::activateFramework ( const FrameworkID &  frameworkId)
inlinevirtual

Activates a framework in the Mesos cluster.

Offers are only sent to active frameworks.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::activateSlave ( const SlaveID &  slaveId)
inlinevirtual

Activates an agent.

This is invoked when an agent reregisters. Offers are only sent for activated agents.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::addFramework ( const FrameworkID &  frameworkId,
const FrameworkInfo &  frameworkInfo,
const hashmap< SlaveID, Resources > &  used,
bool  active,
const std::set< std::string > &  suppressedRoles 
)
inlinevirtual

Adds a framework to the Mesos cluster.

The allocator is invoked when a new framework joins the Mesos cluster and is entitled to participate in resource sharing.

Parameters
usedResources used by this framework. The allocator should account for these resources when updating the allocation of this framework. The allocator should avoid double accounting when yet unknown agents are added later in addSlave().
activeWhether the framework is initially activated.
suppressedRolesList of suppressed roles for this framework.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::addResourceProvider ( const SlaveID &  slave,
const Resources total,
const hashmap< FrameworkID, Resources > &  used 
)
virtual

Add resources from a local resource provider to an agent.

Parameters
slaveId of the agent to modify.
totalThe resources to add to the agent's total resources.
usedThe resources to add to the resources tracked as used for this agent.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::addSlave ( const SlaveID &  slaveId,
const SlaveInfo &  slaveInfo,
const std::vector< SlaveInfo::Capability > &  capabilities,
const Option< Unavailability > &  unavailability,
const Resources total,
const hashmap< FrameworkID, Resources > &  used 
)
inlinevirtual

Adds or re-adds an agent to the Mesos cluster.

It is invoked when a new agent joins the cluster or in case of agent recovery.

Parameters
slaveIdID of the agent to be added or re-added.
slaveInfoDetailed info of the agent. The slaveInfo resources correspond directly to the static –resources flag value on the agent.
capabilitiesCapabilities of the agent.
totalThe total resources are passed explicitly because it includes resources that are dynamically "checkpointed" on the agent (e.g. persistent volumes, dynamic reservations, etc).
usedResources that are allocated on the current agent. The allocator should avoid double accounting when yet unknown frameworks are added later in addFramework().

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
Try< mesos::allocator::Allocator * > mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::create ( )
static
template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::deactivateFramework ( const FrameworkID &  frameworkId)
inlinevirtual

Deactivates a framework in the Mesos cluster.

Resource offers are not sent to deactivated frameworks.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::deactivateSlave ( const SlaveID &  slaveId)
inlinevirtual

Deactivates an agent.

This is triggered if an agent disconnects from the master. The allocator should treat all offers from the deactivated agent as rescinded. (There is no separate call to the allocator to handle this). Resources aren't "recovered" when an agent deactivates because the resources are lost.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
process::Future< hashmap< SlaveID, hashmap< FrameworkID, mesos::allocator::InverseOfferStatus > > > mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::getInverseOfferStatuses ( )
inlinevirtual

Retrieves the status of all inverse offers maintained by the allocator.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::initialize ( const Duration allocationInterval)
inlinevirtual

Initializes the allocator when the master starts up.

Any errors in initialization should fail fast and result in an ABORT. The master expects the allocator to be successfully initialized if this call returns.

Parameters
allocationIntervalThe allocate interval for the allocator, it determines how often the allocator should perform the batch allocation. An allocator may also perform allocation based on events (a framework is added and so on), this depends on the implementation.
offerCallbackA callback the allocator uses to send allocations to the frameworks.
inverseOfferCallbackA callback the allocator uses to send reclaim allocations from the frameworks.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::recover ( const int  expectedAgentCount,
const hashmap< std::string, Quota > &  quotas 
)
inlinevirtual

Informs the allocator of the recovered state from the master.

Because it is hard to define recovery for a running allocator, this method should be called after initialize(), but before actual allocation starts (i.e. addSlave() is called).

TODO(alexr): Consider extending the signature with expected frameworks count once it is available upon the master failover.

Parameters
quotasA (possibly empty) collection of quotas, keyed by their role, known to the master.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::recoverResources ( const FrameworkID &  frameworkId,
const SlaveID &  slaveId,
const Resources resources,
const Option< Filters > &  filters 
)
inlinevirtual

Recovers resources.

Used to update the set of available resources for a specific agent. This method is invoked to inform the allocator about allocated resources that have been refused or are no longer in use. Allocated resources will have an allocation_info.role assigned and callers are expected to only call this with resources allocated to a single role.

TODO(bmahler): We could allow resources allocated to multiple roles within a single call here, but filtering them in the same way does not seem desirable.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::removeFramework ( const FrameworkID &  frameworkId)
inlinevirtual

Removes a framework from the Mesos cluster.

It is up to an allocator to decide what to do with framework's resources. For example, they may be released and added back to the shared pool of resources.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::removeQuota ( const std::string &  role)
inlinevirtual

Informs the allocator to remove quota for the given role.

An allocator implementation may employ different strategies for roles with or without quota. Hence an empty (or zero) quota is not necessarily the same as an absence of quota. Logically, this method implies that the given role should be transitioned to the group of roles without quota set (absence of quota). An allocator implementation may assert quota for the given role is set prior to the call and react accordingly if this assumption is violated (i.e. fail).

TODO(alexr): Consider returning a future which an allocator can fail in order to report failure.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::removeSlave ( const SlaveID &  slaveId)
inlinevirtual

Removes an agent from the Mesos cluster.

All resources belonging to this agent should be released by the allocator.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::requestResources ( const FrameworkID &  frameworkId,
const std::vector< Request > &  requests 
)
inlinevirtual

Requests resources for a framework.

A framework may request resources via this call. It is up to the allocator how to react to this request. For example, a request may be ignored, or may influence internal priorities the allocator may keep for frameworks.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::reviveOffers ( const FrameworkID &  frameworkId,
const std::set< std::string > &  roles 
)
inlinevirtual

Revives offers to this framework for the specified roles.

This is invoked by a framework when it wishes to receive filtered resources immediately or get itself out of a suppressed state. If roles is an empty set, it is treated as being set to all of the framework's subscribed roles.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::setQuota ( const std::string &  role,
const Quota quota 
)
inlinevirtual

Informs the allocator to set quota for the given role.

It is up to the allocator implementation how to satisfy quota. An implementation may employ different strategies for roles with or without quota. Hence an empty (or zero) quota is not necessarily the same as an absence of quota. Logically, this method implies that the given role should be transitioned to the group of roles with quota set. An allocator implementation may assert quota for the given role is not set prior to the call and react accordingly if this assumption is violated (i.e. fail).

TODO(alexr): Consider returning a future which an allocator can fail in order to report failure.

TODO(alexr): Consider adding an updateQuota() method which allows updating existing quota.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::suppressOffers ( const FrameworkID &  frameworkId,
const std::set< std::string > &  roles 
)
inlinevirtual

Suppresses offers.

Informs the allocator to stop sending offers to this framework for the specified roles. If roles is an empty set, we will stop sending offers to this framework for all of the framework's subscribed roles.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::updateAllocation ( const FrameworkID &  frameworkId,
const SlaveID &  slaveId,
const Resources offeredResources,
const std::vector< ResourceConversion > &  conversions 
)
inlinevirtual

Updates allocation by applying offer operations.

This call is mainly intended to support persistence-related features (dynamic reservation and persistent volumes). The allocator may react differently for certain offer operations. The allocator should use this call to update bookkeeping information related to the framework. The offeredResources are the resources that the operations are applied to and must be allocated to a single role.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
process::Future< Nothing > mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::updateAvailable ( const SlaveID &  slaveId,
const std::vector< Offer::Operation > &  operations 
)
inlinevirtual

Updates available resources on an agent based on a sequence of offer operations.

Operations may include reserve, unreserve, create or destroy.

Parameters
slaveIdID of the agent.
operationsThe offer operations to apply to this agent's resources.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::updateFramework ( const FrameworkID &  frameworkId,
const FrameworkInfo &  frameworkInfo,
const std::set< std::string > &  suppressedRoles 
)
inlinevirtual

Updates capabilities of a framework in the Mesos cluster.

This will be invoked when a framework is re-added. As some of the framework's capabilities may be updated when re-added, this API should update the capabilities of the newly added framework to Mesos cluster to reflect the latest framework info. Please refer to the design document here https://cwiki.apache.org/confluence/display/MESOS/Design+doc:+Updating+Framework+Info // NOLINT for more details related to framework update.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::updateInverseOffer ( const SlaveID &  slaveId,
const FrameworkID &  frameworkId,
const Option< UnavailableResources > &  unavailableResources,
const Option< mesos::allocator::InverseOfferStatus > &  status,
const Option< Filters > &  filters 
)
inline
template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::updateSlave ( const SlaveID &  slave,
const SlaveInfo &  slaveInfo,
const Option< Resources > &  total = None(),
const Option< std::vector< SlaveInfo::Capability >> &  capabilities = None() 
)
inlinevirtual

Updates an agent.

TODO(bevers): Make total and capabilities non-optional.

Parameters
slaveInfoThe current slave info of the agent.
totalThe new total resources on the agent.
capabilitiesThe new capabilities of the agent.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::updateUnavailability ( const SlaveID &  slaveId,
const Option< Unavailability > &  unavailability 
)
inlinevirtual

Updates unavailability for an agent.

We currently support storing the next unavailability, if there is one, per agent. If unavailability is not set then there is no known upcoming unavailability. This might require the implementation of the function to remove any inverse offers that are outstanding.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::updateWeights ( const std::vector< WeightInfo > &  weightInfos)
inlinevirtual

Updates the weight associated with one or more roles.

If a role was previously configured to have a weight and that role is omitted from this list, it keeps its old weight.

Implements mesos::allocator::Allocator.

template<typename AllocatorProcess >
void mesos::internal::master::allocator::MesosAllocator< AllocatorProcess >::updateWhitelist ( const Option< hashset< std::string >> &  whitelist)
inlinevirtual

Updates the list of trusted agents.

This is invoked when the master starts up with the –whitelist flag.

Parameters
whitelistA set of agents that are allowed to contribute their resources to the resource pool.

Implements mesos::allocator::Allocator.


The documentation for this class was generated from the following file: