Providing In-network Support to Coflow Scheduling

• Many emerging distributed applications, including big data analytics, generate a number of flows that concurrently transport data across data center networks. To improve their performance, it is required to account for the behavior of a collection of flows, i.e., coflows, rather than individual. State-of-the-art solutions allow for a near-optimal completion time by continuously reordering the unfinished coflows at the end-host, using network priorities. This paper shows that dynamically changing flow priorities at the end host, without taking into account in-flight packets, can cause high-degrees of packet re-ordering, thus imposing pressure on the congestion control and potentially harming network performance in the presence of switches with shallow buffers. We present pCoflow, a new solution that integrates end-host based coflow ordering with in-network scheduling based on packet history. Our evaluation shows that pCoflow improves in CCT upon state-of-the-art solutions by up to 34% for varying load.

Subject headings

NATURVETENSKAP  -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Computer Sciences (hsv//eng)

Keyword

Coflow

Datacenter Networks

P4

Dataplane Programming

Computer Science

Datavetenskap

Publication and Content Type

ref (subject category)

kon (subject category)