Manage nodes

Reserve GPU nodes so that a team's new jobs run only on those nodes. Manage reservations and view node inventory in Settings > Nodes.

Prerequisites

Before you start, make sure that you have the following.

• The platform administrator role. See Teams and roles.

Understand node reservations

• A team with reserved nodes runs its new jobs only on those nodes.
• Only one team can reserve a node at a time.
• A team without a reservation follows the policy for unreserved nodes. The Unreserved nodes card at the top of the Nodes page shows the policy.
  • Shared: teams without a reservation share the nodes that no team has reserved. Its value is shared, the default.
  • Blocked: teams without a reservation can't run jobs. Its value is deny.

To set the policy, set WRKORK_COMMON_CONFIG__node_reservations__default_policy in .env when you deploy CosmicAC. You can't change the policy on the Nodes page. For more information, see Service settings.

Open the nodes page

1. In the sidebar, click Settings.
2. In the Instance group, click Nodes.

The Reservations tab lists the teams that have reserved nodes. The Inventory tab lists the nodes that online racks report.

View node inventory

1. Open the Inventory tab.
2. Find the node you want to check.

For each node, the inventory shows its rack, GPU type, Free GPUs, and the team in Reserved by.

Reserve nodes for a team

1. On the Nodes page, click Reserve nodes.
2. In the Team list, select the team.
3. Select the nodes to reserve. You can't select a node that shows Held by, because another team already holds the node.
4. Click Save changes.

A Nodes updated message confirms the reservation.

You can also manage a team's reservation from Settings > Teams. In the team's row, click Manage nodes.

Change a team's reservation

1. On the Reservations tab, click the team's row.
2. Select the nodes to add and clear the nodes to release.
3. Click Save changes.

To release all reserved nodes, clear every selected node, and then click Release all nodes. In the confirmation dialog, click Release all nodes.

Clear a team's reservation

1. On the Reservations tab, open the menu for the team.
2. Click Clear reservation.
3. In the Clear reservation dialog, click Clear reservation.

Help and troubleshooting

Saving fails because another team already holds a selected node

Only one team can reserve a node at a time, and another team holds one of the selected nodes. To find which team, check the node's Reserved by value on the Inventory tab. Remove the node from your selection, or release it from the other team's reservation first.

Next steps

• Manage teams
• Manage team members
• Settings pages and access