WO2013123846A1 - Distributed network control method and device - Google Patents

Distributed network control method and device Download PDF

Info

Publication number
WO2013123846A1
WO2013123846A1 PCT/CN2013/071151 CN2013071151W WO2013123846A1 WO 2013123846 A1 WO2013123846 A1 WO 2013123846A1 CN 2013071151 W CN2013071151 W CN 2013071151W WO 2013123846 A1 WO2013123846 A1 WO 2013123846A1
Authority
WO
WIPO (PCT)
Prior art keywords
network
information
controller
switch
distributed
Prior art date
Application number
PCT/CN2013/071151
Other languages
French (fr)
Chinese (zh)
Inventor
胡永生
汪军
李漓春
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2013123846A1 publication Critical patent/WO2013123846A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/04Network management architectures or arrangements
    • H04L41/044Network management architectures or arrangements comprising hierarchical management structures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/12Discovery or management of network topologies

Definitions

  • the present invention relates to the field of computer network technologies, and in particular, to a distributed network control method and apparatus. Background technique
  • the external control plane entity uses the OpenFlow protocol to control the forwarding plane device to implement various forwarding logics.
  • the main function of the forwarding plane device is based on control.
  • the forwarding flow table delivered by the polygon entity performs controlled forwarding, and its behavior is standardized: the forwarding plane device receives a packet, and extracts the L2/L3/L4 related field values of the packet header, and uses these related field values as the key.
  • the word search forwarding flow table is matched to the entry, and the message field is transformed according to the instruction set in the content of the entry. After the transformation is completed, the flow is forwarded to a logical or physical port according to the indication.
  • the OpenFlow protocol is further evolved into the foundation of Software Defined Network (SDN) technology, which can implement various complex network applications by using software programming on the control plane entity, without any change in the forwarding plane device.
  • the control plane device uses a general-purpose server + general-purpose operating system, and can be implemented using a common software programming tool or a scripting language such as Python, which makes the support of the new network protocol very simple, and the new technology deployment cycle is greatly shorten.
  • a basic OpenFlow network consists of two main control entities, the network controller and the switch.
  • a centralized network controller is deployed in the network responsible for the management domain control of all OpenFlow switches throughout the network domain.
  • a single centralized network controller can become a bottleneck for network scale.
  • the main object of the present invention is to provide a distributed network control method and device, which can solve the performance bottleneck problem of the network controller.
  • the present invention provides a distributed network control method, in which a plurality of controller nodes form a distributed controller network, and each controller node manages one or more switch devices;
  • the controller node receives the information reported by the switch, synchronizes network topology change information in the information between all controller nodes, and saves the host location information in the information to the distributed control network;
  • the controller node When receiving the data stream processing request, the controller node queries the distributed controller network for the destination address information of the data stream, and establishes a data exchange path for the data stream according to the locally stored global network topology information.
  • the network topology change information is synchronized between all controller nodes: the controller node synchronizes the network topology change information between all controller nodes by using an independent global event synchronization system, or utilizes The message broadcast mechanism of the distributed controller network synchronizes the network topology change information across all controller nodes.
  • the method further includes: the controller node storing the synchronized network topology change information, and obtaining consistent global network topology information according to the network topology change information;
  • the network topology change information is: attributes of the switch, or a link state of the switch joining the network, or the switch leaving the network, or the switch.
  • the saving the host location information to the distributed control network is: the controller node saves the host location information to a controller node other than the local node in the distributed controller network;
  • the method further includes: the controller node receiving the information reported by the switch simultaneously saves the received host location information locally.
  • the host location information is: a data link layer location and/or a network layer location of a host connected to the switch.
  • the data link layer location is a correspondence between a MAC address of a host connected to the switch and a port identifier of the switch;
  • the network layer location is a correspondence between an IP address of a host connected to the switch and a port identifier of the switch, or a correspondence between the network domain identifier and a port identifier of the switch.
  • the destination address information of the data stream in the distributed controller network is:
  • the controller node After receiving the data stream processing request, the controller node first queries the node for the destination address information of the data stream, and then queries the controller node other than the local node in the distributed control network to query the destination address information of the data stream.
  • the destination address information is: a port identifier of a switch corresponding to a host MAC address and/or an IP address, or a port list information of a switch corresponding to the network domain identifier.
  • the present invention also provides a distributed network control device, which is a controller node, and the controller node includes: a control module, an event synchronization module, and a distributed processing module;
  • the control module is configured to receive the information reported by the switch, and is further configured to receive the data stream processing request, and after the distributed processing module queries the destination address of the data stream, establish data for the data stream according to the locally stored global network topology information.
  • An event synchronization module configured to synchronize network topology change information in the information reported by the switch between all controller nodes; a distributed processing module, configured to save host location information in the information reported by the switch to the distributed control network; and, after receiving the data stream processing request, querying the data flow in the distributed controller network Address information.
  • the device further comprises:
  • the information storage module is configured to save synchronized network topology change information, and/or received host location information, and obtain consistent global network topology information according to the network topology change information.
  • the controller node receives the information reported by the switch, synchronizes the network topology change information in the information between all controller nodes, and saves the host location information in the information. Going to the distributed control network; when receiving the data stream processing request, the controller node queries the distributed controller network for the destination address information of the data stream, and uses the locally stored global network topology information as the data stream. The data exchange path is established. Therefore, in the present invention, a small amount of necessary network topology information is globally synchronized, and the host related information that is frequently changed is distributed and queried by the controller node, thereby being able to adapt to the linear growth of the network scale. Solve performance bottlenecks in network controllers. DRAWINGS
  • FIG. 1 is a schematic flow chart of a method for implementing a distributed network control according to the present invention
  • FIG. 2 is a diagram showing an example of networking for implementing a distributed network control method according to the present invention
  • FIG. 3 is a schematic flowchart of a specific implementation method of the implementation step 101 of the present invention.
  • FIG. 4 is a schematic flow chart of a specific implementation method of the implementation step 106 of the present invention.
  • FIG. 5 is a schematic flowchart diagram of a specific implementation method of the implementation step 103 of the present invention.
  • FIG. 6 is a schematic structural diagram of a distributed network control apparatus according to the present invention. detailed description
  • the basic idea of the present invention is that the controller node receives the information reported by the switch, synchronizes the network topology change information among all controller nodes, and maintains the host location information therein. And storing, in the distributed control network, the controller node, when receiving the data stream processing request, querying the destination address information of the data flow in the distributed controller network, and using the locally stored global network topology information as the data The flow establishes a data exchange path.
  • FIG. 1 is a schematic flowchart of a distributed network control method according to the present invention. As shown in FIG. 1, the method includes the following steps:
  • Step 101 The controller node receives the network topology change information reported by the switch, and synchronizes the network topology change information between all controller nodes.
  • a plurality of controller nodes form a distributed controller network, which is a distributed network, such as a distributed hash table (DHT) of One-hop.
  • DHT distributed hash table
  • each controller node manages one or more switch devices, and multiple switches are interconnected by physical links and form a switch network;
  • the controller node receives the network topology change information reported by the switch managed by the switch, and the network topology change information includes: attributes of the switch, the switch joins the network, the switch leaves the network, and the link state between the switches; the controller node is at
  • the network topology change information is synchronized in all controller nodes in the distributed controller network, and the other controller nodes save the synchronized network topology change information, so that all controller nodes can be consistent according to the network topology change information.
  • Global network topology information is synchronized in all controller nodes in the distributed controller network, and the other controller nodes save the synchronized network topology change information, so that all controller nodes can be consistent according to the network topology change information.
  • the method for synchronizing may be through an independent global event synchronization system, for example: an event publishing system, an event subscription system, or a distributed controller network, for example: a distributed control message broadcast mechanism of its network.
  • an independent global event synchronization system for example: an event publishing system, an event subscription system, or a distributed controller network, for example: a distributed control message broadcast mechanism of its network.
  • Step 102 The controller node receives the host location information reported by the switch, and saves the host location information to the distributed controller network.
  • the controller node receives the host location information reported by the switch managed by the controller, where the host location information includes: a data link layer location and/or a network layer connected to the host on the switch.
  • the data link layer location refers to the medium access control of the host connected to the switch.
  • the network layer location refers to the correspondence between the IP address of the host connected to the switch and the port identifier of the switch, or the correspondence between the network domain identifier and the port identifier of the switch. Relationship;
  • the controller node saves the host location information to other controller nodes in the distributed controller network. Preferably, the received host location information can also be saved locally at the same time.
  • Step 103 When receiving the data stream processing request, the controller node queries the distributed controller network for the destination address information of the data flow, and establishes a data exchange path for the data flow according to the locally stored global network topology information.
  • the controller node when receiving the data stream processing request, queries the controller node in the distributed controller network for the destination address information of the data stream, where the controller node includes the controller node that receives the data stream processing request, And other controller nodes in the distributed control network; in order to improve the query speed, after receiving the data stream processing request, the controller node may first query the destination address information of the data stream in the node, and then in the distributed control network.
  • the other controller node queries the destination address information of the data stream; the destination address information may be the port identifier of the switch corresponding to the host MAC address and/or the IP address, or the port list information of the switch corresponding to the network domain identifier.
  • the controller node selects a switching path for the data stream according to the destination address information of the queried data stream and the global network topology information stored locally, and sends a corresponding forwarding flow table to the switch device on the switching path, thereby implementing The data stream establishes a data exchange path.
  • Step 301 The switch SW1 joins the OpenFlow network.
  • the process of joining the OpenFlow network includes: establishing a connection with the controller node A, and performing related security verification, basic information reporting, and switch configuration operations.
  • Step 302 Since the switch SW1 joins the OpenFlow network, the controller node A issues a network event that the switch SW1 joins to the global event synchronization system.
  • the global event synchronization system is an independent event subscription system or an event issuance system; the network event that the switch SW1 joins needs to carry basic information of the switch SW1, such as the identifier of the switch SW1, port configuration information, and the like.
  • Step 303 The global event synchronization system synchronizes network events added by the switch SW1 in the distributed controller network, and other controller nodes in the distributed controller network learn that the switch SW1 joins the network under the controller node A.
  • Step 304 The switch SW1 reports the received link detection packet to the controller node A, where the network topology change information is carried.
  • Step 305 The controller node A determines, according to the link detection message, that a new link exists between the switch SW1 and another switch SWX.
  • the new link may be a link that is not originally discovered, or a link.
  • the state of the road changes.
  • Step 306 the controller node A issues a link network event between the switch SW1 and the switch SWX to the global event synchronization system;
  • the link network event between the switch S W1 and the switch SWX needs to carry basic information about the link, such as the switch identifier of the switch SW1 and the switch SWX, the switch port identifier where the link is located, and the bandwidth of the link. information.
  • Step 307 The global event synchronization system synchronizes the link network events of the switch SW1 and the switch SWX in the distributed controller network, and the other controller nodes in the distributed controller network learn the link between the switch SW1 and the switch SWX. A change has occurred.
  • Step 308 controller node A finds that switch SW1 leaves the network. Step 309, the controller node A issues a network event that the switch SW1 leaves to the global event synchronization system;
  • the network event that the switch SW1 leaves needs to carry the basic information of the switch SW1, such as the identifier of the switch.
  • Step 310 The global event synchronization system synchronizes the network events that the switch SW1 leaves in the distributed controller network, and the other controller nodes in the distributed controller network learn that the switch SW1 leaves the network under the controller node A, and updates each Stored global network topology information.
  • the synchronization of network topology change events between all controller nodes by using the global event synchronization system can also be implemented by a message broadcast mechanism of the distributed controller network.
  • the controller node learns the host location information from the data packet reported by the switch, and stores the host location information in the distributed controller network, as shown in FIG. 4 .
  • the method includes the following steps:
  • Step 401 The host sends a data packet to the switch SW1 connected to the host.
  • Step 402 The switch SW1 receives the data packet, and after querying the local forwarding flow table, it finds that there is no corresponding entry, and therefore reports the data packet to the controller node A.
  • Step 403 The controller node A checks the source address of the received data packet. If the source address of the data packet is a new host or the address of the corresponding host changes, the host location information needs to be updated.
  • controller node A may choose to store the host location information locally.
  • Step 404 the controller node A stores the host location information on the controller node X of the distributed controller network, and the specific storage manner is related to the mechanism of the distributed controller network, for example, the distributed controller network is a single-hop distributed For a One-hop DHT network, controller node A needs to hash the identity of the host and store it in the controller node.
  • the host location information that needs to be stored is the MAC of the host connected to the switch.
  • the correspondence between the address and the port identifier of the switch, or the correspondence between the IP address of the host connected to the switch and the port identifier of the switch, or the correspondence between the network domain identifier and the port identifier of the switch; the network domain identifier includes the VLAN identifier, MPLS logo, etc.
  • the host location information that needs to be stored may be stored in one or more indexes, for example, by using the host's MAC address, and/or IP address, and/or the network domain identifier as a key for storage. .
  • the host A connected to the switch SW1 is to communicate with the host B connected to the switch SW2, and the first data packet is sent by the switch SW1 to the controller node A.
  • the controller node A queries the destination address information of the data packet through the distributed controller network, and selects and establishes an appropriate switching path according to the global network topology information. As shown in FIG. 5, the method includes the following steps:
  • Step 501 Host A wants to communicate with host B, and sends the first data packet to the connected switch SW1.
  • Step 502 After the switch SW1 queries the local forwarding flow table that there is no match, the switch reports the data packet to the controller node A, and is used to initiate a data flow processing request to the controller node A.
  • Step 503 The controller node A queries the other controller nodes in the distributed controller network for the destination address information of the data packet.
  • the destination address information of the data message may be an address list.
  • the controller node A does not need to search for other controller nodes in the distributed controller network. Destination address information of the data packet.
  • Step 504 The controller node A selects a switching path for the data packet according to the source and destination address information of the data packet and the global network topology information stored in the local network.
  • the switching path in this embodiment is the switch SW1 and Switch SW2.
  • Step 505 The controller node A sends the forwarding flow table of the data packet to the switch SW1.
  • step 506 the controller node A sends the forwarding flow table of the data packet to the switch SW2.
  • the step 504 is directly connected to the controller node B, the step is:
  • the controller node A sends the forwarding flow table to the controller node B, and the controller node B delivers the forwarding flow table to the switch SW2.
  • Step 507 The switch SW1 forwards the data packet according to the received forwarding flow table, where the switch forwards to the switch SW2.
  • Step 508 The switch SW2 forwards the data packet according to the received forwarding flow table, where the forwarding is performed to the connected host B.
  • the present invention further provides a distributed network control device, which is a controller node
  • FIG. 6 is a schematic structural diagram of a distributed network control device according to the present invention.
  • the device includes: Module 61, event synchronization module 62, distributed processing module 63; wherein
  • the control module 61 is configured to receive the information reported by the switch, and is further configured to receive the data stream processing request, and after the distributed processing module queries the destination address of the data stream, according to the locally stored global network topology information, the data stream is used as the data stream. Establish a data exchange path;
  • the event synchronization module 62 is configured to synchronize network topology change information in the reported information between all controller nodes;
  • the distributed processing module 63 is configured to save the host location information in the reported information to the distributed control network, and is further configured to: after receiving the data stream processing request, query the data flow in the distributed controller network. Address information.
  • the device also includes:
  • the information storage module 64 is configured to save the synchronized network topology change information, and/or the received host location information, and obtain consistent global network topology information according to the network topology change information.
  • the event synchronization module 62 will use the network topology change information in all controller sections.
  • Inter-point synchronization is: synchronizing the network topology change information between all controller nodes by using a separate global event synchronization system, or synchronizing the network topology among all controller nodes by using a message broadcast mechanism of a distributed controller network Change information.
  • the network topology change information is: an attribute of the switch, or a link state of the switch joining the network, or the switch leaving the network, or the switch.
  • the distributed processing module 63 saves the host location information to the distributed control network as: saving the host location information to other controller nodes in the distributed controller network?
  • the host location information is: a data link layer location and/or a network layer location of a host connected to the switch.
  • the data link layer location is a correspondence between a MAC address of a host connected to the switch and a port identifier of the switch;
  • the network layer location is a correspondence between an IP address of the host connected to the switch and a port identifier of the switch, or a network The relationship between the domain ID and the port ID of the switch.
  • the destination address information of the distributed processing module 63 for querying the data flow in the distributed controller network is: after receiving the data flow processing request, first querying the destination address information of the data flow in the node, and then distributing The other controller nodes in the control network query the destination address information of the data stream.
  • the destination address information is: a port identifier of a switch corresponding to the host MAC address and/or the IP address, or a port list information of the switch corresponding to the network domain identifier.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

Disclosed is a distributed network control method. A plurality of controller nodes form a distributed controller network, and each controller node manages one or more exchanger devices. The method comprises: controller nodes receiving information reported by an exchanger, synchronizing network topology change information in the information among all the controller nodes, and storing host position information in the information into a distributed controller network; and when receiving a data stream processing request, the controller nodes querying destination address information about a data stream in the distributed controller network, and establishing a data exchange path for the data stream according to locally stored global network topology information. Also provided is a distributed network control device. According to the technical solution of the present invention, the performance bottleneck problem of a network controller can be solved.

Description

一种分布式网络控制方法及装置 技术领域  Distributed network control method and device
本发明涉及计算机网络技术领域, 尤其涉及一种分布式网络控制方法 及装置。 背景技术  The present invention relates to the field of computer network technologies, and in particular, to a distributed network control method and apparatus. Background technique
美国斯坦福大学于 2008年提出了 OpenFlow协议, 该协议采用转发 /控 制分离架构, 外置控制面实体利用 OpenFlow协议控制转发面设备, 以实现 各种转发逻辑, 而转发面设备的主要功能就是根据控制面实体下发的转发 流表执行受控转发, 其行为是标准化的: 转发面设备收到一条报文, 取出 该报文头部 L2/L3/L4相关字段值, 将这些相关字段值作为关键字查找转发 流表, 匹配到表项后, 根据表项内容中的指令集对报文字段进行变换, 变 换完毕后根据指示转发到某一逻辑或物理端口。 OpenFlow协议进一步演进 为软件定义网络(SDN, Software Defined Network )技术的基础, 可以在控 制面实体采用软件编程实现各种复杂的网络应用, 而转发面设备无需任何 改变。 控制面设备采用通用服务器+通用操作系统, 并且可以使用通用的软 件编程工具, 也可以使用 Python这样的脚本编程语言实现, 这使得新的网 络协议的支持变得非常简单, 而且新技术部署周期大大缩短。 一个基本的 OpenFlow网络包括网络控制器和交换机两个主要的控制实体。 通常地, 在 网络中部署一个集中的网络控制器,负责整个网络域中所有的 OpenFlow交 换机的管理域控制。 然而, 单个集中式的网络控制器会成为网络规模化的 瓶颈, 如与网络控制器距离较远的交换机建立流的时延增加, 单个网络控 制器处理交换路径请求的吞吐量受限, 端到端的路径带宽可控性差等。 为 此, 学术界提出分布式网络控制器方案, 如 HyperFlow、 在线信息交换 ( ONIX )等, 但这些方案一般是在网络上部署多个网络控制器, 通过事件 消息同步或存储共享等方案, 实现状态一致的多个网络控制器共同管理与 控制 OpenFlow网络。对于大规模的网络, 上述方案中需要在网络控制器之 间同步的网络事件每秒有几万个, 现有的全局事件同步机制无法满足需求, 即现有的解决方案不适用大规模网络部署。 发明内容 In 2008, Stanford University proposed the OpenFlow protocol, which uses a forwarding/control separation architecture. The external control plane entity uses the OpenFlow protocol to control the forwarding plane device to implement various forwarding logics. The main function of the forwarding plane device is based on control. The forwarding flow table delivered by the polygon entity performs controlled forwarding, and its behavior is standardized: the forwarding plane device receives a packet, and extracts the L2/L3/L4 related field values of the packet header, and uses these related field values as the key. The word search forwarding flow table is matched to the entry, and the message field is transformed according to the instruction set in the content of the entry. After the transformation is completed, the flow is forwarded to a logical or physical port according to the indication. The OpenFlow protocol is further evolved into the foundation of Software Defined Network (SDN) technology, which can implement various complex network applications by using software programming on the control plane entity, without any change in the forwarding plane device. The control plane device uses a general-purpose server + general-purpose operating system, and can be implemented using a common software programming tool or a scripting language such as Python, which makes the support of the new network protocol very simple, and the new technology deployment cycle is greatly shorten. A basic OpenFlow network consists of two main control entities, the network controller and the switch. Typically, a centralized network controller is deployed in the network responsible for the management domain control of all OpenFlow switches throughout the network domain. However, a single centralized network controller can become a bottleneck for network scale. For example, the delay of establishing a flow of a switch that is far away from the network controller increases, and the throughput of a single network controller processing the exchange path request is limited. The path bandwidth of the end is poorly controllable. To this end, the academic community has proposed distributed network controller solutions, such as HyperFlow, online information exchange. (ONIX), etc., but these solutions generally deploy multiple network controllers on the network, and implement multiple states of the same network controller to manage and control the OpenFlow network through event message synchronization or storage sharing. For large-scale networks, there are tens of thousands of network events that need to be synchronized between network controllers in the above solution. The existing global event synchronization mechanism cannot meet the requirements, that is, the existing solution is not suitable for large-scale network deployment. . Summary of the invention
有鉴于此, 本发明的主要目的在于提供一种分布式网络控制方法及装 置, 可解决网络控制器的性能瓶颈问题。  In view of this, the main object of the present invention is to provide a distributed network control method and device, which can solve the performance bottleneck problem of the network controller.
为达到上述目的, 本发明的技术方案是这样实现的:  In order to achieve the above object, the technical solution of the present invention is achieved as follows:
本发明提供一种分布式网络控制方法, 多个控制器节点组成分布式控 制器网络, 每个控制器节点管理一个或多个交换机设备; 包括:  The present invention provides a distributed network control method, in which a plurality of controller nodes form a distributed controller network, and each controller node manages one or more switch devices;
控制器节点接收交换机上报的信息 , 将所述信息中的网络拓朴变化信 息在所有控制器节点间同步, 将所述信息中的主机位置信息保存到分布式 控制网络中;  The controller node receives the information reported by the switch, synchronizes network topology change information in the information between all controller nodes, and saves the host location information in the information to the distributed control network;
所述控制器节点收到数据流处理请求时, 在分布式控制器网络中查询 数据流的目的地址信息, 并根据本地存储的全局网络拓朴信息为所述数据 流建立数据交换路径。  When receiving the data stream processing request, the controller node queries the distributed controller network for the destination address information of the data stream, and establishes a data exchange path for the data stream according to the locally stored global network topology information.
上述方法中, 所述将网络拓朴变化信息在所有控制器节点间同步为: 所述控制器节点利用独立的全局事件同步系统在所有控制器节点间同 步所述网络拓朴变化信息, 或利用分布式控制器网络的消息广播机制在所 有控制器节点间同步所述网络拓朴变化信息。  In the above method, the network topology change information is synchronized between all controller nodes: the controller node synchronizes the network topology change information between all controller nodes by using an independent global event synchronization system, or utilizes The message broadcast mechanism of the distributed controller network synchronizes the network topology change information across all controller nodes.
上述方法中, 该方法还包括: 控制器节点保存同步的网络拓朴变化信 息, 并根据所述网络拓朴变化信息得到一致的全局网络拓朴信息;  In the above method, the method further includes: the controller node storing the synchronized network topology change information, and obtaining consistent global network topology information according to the network topology change information;
所述网络拓朴变化信息为: 交换机的属性、 或交换机加入网络、 或交 换机离开网络、 或交换机之间的链路状态。 上述方法中, 所述将主机位置信息保存到分布式控制网络中为: 所述 控制器节点将所述主机位置信息保存到分布式控制器网络中的除本节点之 外的控制器节点中; The network topology change information is: attributes of the switch, or a link state of the switch joining the network, or the switch leaving the network, or the switch. In the above method, the saving the host location information to the distributed control network is: the controller node saves the host location information to a controller node other than the local node in the distributed controller network;
该方法还包括: 收到交换机上报的信息的控制器节点同时在本地保存 收到的主机位置信息。  The method further includes: the controller node receiving the information reported by the switch simultaneously saves the received host location information locally.
上述方法中, 所述主机位置信息为: 连接在交换机上主机的数据链路 层位置和 /或网络层位置。  In the above method, the host location information is: a data link layer location and/or a network layer location of a host connected to the switch.
上述方法中,所述数据链路层位置为连接在交换机上主机的 MAC地址 与交换机的端口标识的对应关系;  In the above method, the data link layer location is a correspondence between a MAC address of a host connected to the switch and a port identifier of the switch;
所述网络层位置为连接在交换机上主机的 IP地址与交换机的端口标识 的对应关系, 或网络域标识与交换机的端口标识的对应关系。  The network layer location is a correspondence between an IP address of a host connected to the switch and a port identifier of the switch, or a correspondence between the network domain identifier and a port identifier of the switch.
上述方法中, 所述在分布式控制器网络中查询数据流的目的地址信息 为:  In the above method, the destination address information of the data stream in the distributed controller network is:
控制器节点在收到数据流处理请求后, 首先在本节点查询数据流的目 的地址信息, 再在分布式控制网络中的除本节点之外的控制器节点查询数 据流的目的地址信息。  After receiving the data stream processing request, the controller node first queries the node for the destination address information of the data stream, and then queries the controller node other than the local node in the distributed control network to query the destination address information of the data stream.
上述方法中, 所述目的地址信息为: 主机 MAC地址和 /或 IP地址对应 的交换机的端口标识, 或网络域标识对应的交换机的端口列表信息。  In the above method, the destination address information is: a port identifier of a switch corresponding to a host MAC address and/or an IP address, or a port list information of a switch corresponding to the network domain identifier.
本发明还提供一种分布式网络控制装置, 该装置为控制器节点, 该控 制器节点包括: 控制模块、 事件同步模块、 分布式处理模块; 其中,  The present invention also provides a distributed network control device, which is a controller node, and the controller node includes: a control module, an event synchronization module, and a distributed processing module;
控制模块, 用于接收交换机上报的信息; 还用于接收数据流处理请求, 并在分布式处理模块查询数据流的目的地址后, 根据本地存储的全局网络 拓朴信息为所述数据流建立数据交换路径;  The control module is configured to receive the information reported by the switch, and is further configured to receive the data stream processing request, and after the distributed processing module queries the destination address of the data stream, establish data for the data stream according to the locally stored global network topology information. Exchange path
事件同步模块, 用于将交换机上报的信息中的网络拓朴变化信息在所 有控制器节点间同步; 分布式处理模块, 用于将交换机上报的信息中的主机位置信息保存到 分布式控制网络中; 还用于收到数据流处理请求后, 在所述分布式控制器 网络中查询数据流的目的地址信息。 An event synchronization module, configured to synchronize network topology change information in the information reported by the switch between all controller nodes; a distributed processing module, configured to save host location information in the information reported by the switch to the distributed control network; and, after receiving the data stream processing request, querying the data flow in the distributed controller network Address information.
上述装置中, 该装置还包括:  In the above device, the device further comprises:
信息存储模块, 用于保存同步的网络拓朴变化信息、 和 /或收到的主机 位置信息, 并根据所述网络拓朴变化信息得到一致的全局网络拓朴信息。  The information storage module is configured to save synchronized network topology change information, and/or received host location information, and obtain consistent global network topology information according to the network topology change information.
本发明提供的分布式网络控制方法及装置, 控制器节点接收交换机上 报的信息,将所述信息中的网络拓朴变化信息在所有控制器节点间同步, 将所述信息中的主机位置信息保存到分布式控制网络中; 所述控制器节 点收到数据流处理请求时, 在分布式控制器网络中查询数据流的目的地 址信息, 并根据本地存储的全局网络拓朴信息为所述数据流建立数据交 换路径, 因此, 本发明中对少量必须的网络拓朴信息进行全局同步, 对 于频繁变动的主机相关信息由控制器节点进行分布式的存储和查询, 因 而能够适应网络规模的线性增长, 解决网络控制器的性能瓶颈问题。 附图说明  The distributed network control method and device provided by the present invention, the controller node receives the information reported by the switch, synchronizes the network topology change information in the information between all controller nodes, and saves the host location information in the information. Going to the distributed control network; when receiving the data stream processing request, the controller node queries the distributed controller network for the destination address information of the data stream, and uses the locally stored global network topology information as the data stream. The data exchange path is established. Therefore, in the present invention, a small amount of necessary network topology information is globally synchronized, and the host related information that is frequently changed is distributed and queried by the controller node, thereby being able to adapt to the linear growth of the network scale. Solve performance bottlenecks in network controllers. DRAWINGS
图 1是本发明实现分布式网络控制方法的流程示意图;  1 is a schematic flow chart of a method for implementing a distributed network control according to the present invention;
图 2是本发明实现分布式网络控制方法的组网示例图;  2 is a diagram showing an example of networking for implementing a distributed network control method according to the present invention;
图 3是本发明实现步驟 101的具体实现方法的流程示意图;  3 is a schematic flowchart of a specific implementation method of the implementation step 101 of the present invention;
图 4是本发明实现步驟 102的具体实现方法的流程示意图;  4 is a schematic flow chart of a specific implementation method of the implementation step 106 of the present invention;
图 5是本发明实现步驟 103的具体实现方法的流程示意图;  FIG. 5 is a schematic flowchart diagram of a specific implementation method of the implementation step 103 of the present invention;
图 6是本发明实现分布式网络控制装置的结构示意图。 具体实施方式  FIG. 6 is a schematic structural diagram of a distributed network control apparatus according to the present invention. detailed description
本发明的基本思想是:控制器节点接收交换机上报的信息,将其中的 网络拓朴变化信息在所有控制器节点间同步, 将其中的主机位置信息保 存到分布式控制网络中; 所述控制器节点收到数据流处理请求时, 在分 布式控制器网络中查询数据流的目的地址信息, 并根据本地存储的全局 网络拓朴信息为所述数据流建立数据交换路径。 The basic idea of the present invention is that the controller node receives the information reported by the switch, synchronizes the network topology change information among all controller nodes, and maintains the host location information therein. And storing, in the distributed control network, the controller node, when receiving the data stream processing request, querying the destination address information of the data flow in the distributed controller network, and using the locally stored global network topology information as the data The flow establishes a data exchange path.
下面通过附图及具体实施例对本发明再做进一步的详细说明。  The invention will be further described in detail below with reference to the drawings and specific embodiments.
本发明提供一种分布式网络控制方法, 图 1 是本发明实现分布式网络 控制方法的流程示意图, 如图 1所示, 该方法包括以下步驟:  The present invention provides a distributed network control method. FIG. 1 is a schematic flowchart of a distributed network control method according to the present invention. As shown in FIG. 1, the method includes the following steps:
步驟 101 ,控制器节点接收交换机上报的网络拓朴变化信息, 并在所有 控制器节点间同步该网络拓朴变化信息;  Step 101: The controller node receives the network topology change information reported by the switch, and synchronizes the network topology change information between all controller nodes.
具体的, 如图 2所示, 本发明中, 多个控制器节点组成分布式控制器 网络, 该分布式控制网络是一种分布式网络, 如 One-hop 的分布式哈希表 ( DHT ) 网络; 每个控制器节点管理一个或多个交换机设备, 多个交换机 之间通过物理链路进行互联并组成交换机网络;  Specifically, as shown in FIG. 2, in the present invention, a plurality of controller nodes form a distributed controller network, which is a distributed network, such as a distributed hash table (DHT) of One-hop. Network; each controller node manages one or more switch devices, and multiple switches are interconnected by physical links and form a switch network;
控制器节点接收自身管理的交换机上报的网络拓朴变化信息, 该网络 拓朴变化信息包括: 交换机的属性、 交换机加入网络、 交换机离开网络、 交换机之间的链路状态等信息; 控制器节点在分布式控制器网络中的所有 控制器节点中对该网络拓朴变化信息进行同步, 其他控制器节点保存同步 的网络拓朴变化信息, 这样, 所有控制器节点根据网络拓朴变化信息可以 得到一致的全局网络拓朴信息。  The controller node receives the network topology change information reported by the switch managed by the switch, and the network topology change information includes: attributes of the switch, the switch joins the network, the switch leaves the network, and the link state between the switches; the controller node is at The network topology change information is synchronized in all controller nodes in the distributed controller network, and the other controller nodes save the synchronized network topology change information, so that all controller nodes can be consistent according to the network topology change information. Global network topology information.
其中, 所述同步的方法可以是通过独立的全局事件同步系统, 例如: 事件发布系统、 事件订阅系统, 也可以通过分布式控制器网络实现, 例如: 分布式控制其网络的消息广播机制。  The method for synchronizing may be through an independent global event synchronization system, for example: an event publishing system, an event subscription system, or a distributed controller network, for example: a distributed control message broadcast mechanism of its network.
步驟 102,控制器节点接收交换机上报的主机位置信息, 并将主机位置 信息保存到分布式控制器网络中;  Step 102: The controller node receives the host location information reported by the switch, and saves the host location information to the distributed controller network.
具体的, 控制器节点接收自身管理的交换机上报的主机位置信息, 该 主机位置信息包括: 连接在交换机上主机的数据链路层位置和 /或网络层位 置等, 所述数据链路层位置指的是连接在交换机上主机的介质访问控制Specifically, the controller node receives the host location information reported by the switch managed by the controller, where the host location information includes: a data link layer location and/or a network layer connected to the host on the switch. The data link layer location refers to the medium access control of the host connected to the switch.
( MAC )地址与交换机的端口标识的对应关系, 所述网络层位置指的是连 接在交换机上主机的 IP地址与交换机的端口标识的对应关系, 或是网络域 标识与交换机的端口标识的对应关系; 控制器节点将主机位置信息保存到 分布式控制器网络中的其他控制器节点中, 优选的, 还可以同时在本地保 存收到的主机位置信息。 Correspondence between the (MAC) address and the port identifier of the switch. The network layer location refers to the correspondence between the IP address of the host connected to the switch and the port identifier of the switch, or the correspondence between the network domain identifier and the port identifier of the switch. Relationship; The controller node saves the host location information to other controller nodes in the distributed controller network. Preferably, the received host location information can also be saved locally at the same time.
步驟 103 ,控制器节点收到数据流处理请求时,在分布式控制器网络中 查询数据流的目的地址信息, 并根据本地存储的全局网络拓朴信息为该数 据流建立数据交换路径;  Step 103: When receiving the data stream processing request, the controller node queries the distributed controller network for the destination address information of the data flow, and establishes a data exchange path for the data flow according to the locally stored global network topology information.
具体的, 控制器节点收到数据流处理请求时, 在分布式控制器网络中 的控制器节点中查询数据流的目的地址信息, 该控制器节点包括收到数据 流处理请求的控制器节点, 以及分布式控制网络中的其它控制器节点; 为 了提高查询速度, 控制器节点在收到数据流处理请求后, 可以首先在本节 点查询数据流的目的地址信息, 然后再在分布式控制网络中的其它控制器 节点查询数据流的目的地址信息;所述目的地址信息可以是主机 MAC地址 和 /或 IP地址对应的交换机的端口标识,也可以是网络域标识对应的交换机 的端口列表信息。  Specifically, when receiving the data stream processing request, the controller node queries the controller node in the distributed controller network for the destination address information of the data stream, where the controller node includes the controller node that receives the data stream processing request, And other controller nodes in the distributed control network; in order to improve the query speed, after receiving the data stream processing request, the controller node may first query the destination address information of the data stream in the node, and then in the distributed control network. The other controller node queries the destination address information of the data stream; the destination address information may be the port identifier of the switch corresponding to the host MAC address and/or the IP address, or the port list information of the switch corresponding to the network domain identifier.
控制器节点根据查询到的数据流的目的地址信息, 以及本地存储的全 局网络拓朴信息为该数据流选择交换路径, 并向交换路径上的交换机设备 下发相应的转发流表, 从而实现为该数据流建立数据交换路径。  The controller node selects a switching path for the data stream according to the destination address information of the queried data stream and the global network topology information stored locally, and sends a corresponding forwarding flow table to the switch device on the switching path, thereby implementing The data stream establishes a data exchange path.
图 3是本发明实现步驟 101 的具体实现方法的流程示意图, 每个控制 器节点管理部分交换机, 为了使控制器节点在为业务选择的交换路径达到 全网最优, 需要每个控制器节点有统一的全局网络拓朴信息, 因此, 每个 控制器节点需要将所感知的网络拓朴事件进行全网同步, 如图 3 所示, 该 方法包括以下步驟: 步驟 301 , 交换机 SWl加入 OpenFlow网络; 3 is a schematic flowchart of a specific implementation method of the implementation step 101 of the present invention, where each controller node manages a part of the switch, in order to make the controller node reach the whole network optimal in the exchange path selected for the service, each controller node needs to have Uniform global network topology information. Therefore, each controller node needs to synchronize the network topology events that are perceived to the entire network. As shown in Figure 3, the method includes the following steps: Step 301: The switch SW1 joins the OpenFlow network.
该加入 OpenFlow网络的过程包括: 与控制器节点 A建立连接, 并进 行相关的安全验证、 基本信息上报、 交换机配置等操作。  The process of joining the OpenFlow network includes: establishing a connection with the controller node A, and performing related security verification, basic information reporting, and switch configuration operations.
步驟 302, 由于有交换机 SW1加入 OpenFlow网络, 控制器节点 A向 全局事件同步系统发布交换机 SW1加入的网络事件;  Step 302: Since the switch SW1 joins the OpenFlow network, the controller node A issues a network event that the switch SW1 joins to the global event synchronization system.
全局事件同步系统是一种独立的事件订阅系统或事件发布系统; 所述 交换机 SW1加入的网络事件需要携带交换机 SW1 的基本信息, 例如交换 机 SW1的标识、 端口配置信息等。  The global event synchronization system is an independent event subscription system or an event issuance system; the network event that the switch SW1 joins needs to carry basic information of the switch SW1, such as the identifier of the switch SW1, port configuration information, and the like.
步驟 303 , 全局事件同步系统将交换机 SW1加入的网络事件在分布式 控制器网络中进行同步, 分布式控制器网络中的其它控制器节点获知控制 器节点 A下有交换机 SW1加入网络。  Step 303: The global event synchronization system synchronizes network events added by the switch SW1 in the distributed controller network, and other controller nodes in the distributed controller network learn that the switch SW1 joins the network under the controller node A.
步驟 304, 交换机 SW1将收到的链路探测报文上报给控制器节点 A, 其中携带网络拓朴变化信息。  Step 304: The switch SW1 reports the received link detection packet to the controller node A, where the network topology change information is carried.
步驟 305 , 控制器节点 A根据链路探测报文判断出交换机 SW1与另一 个交换机 SWX间存在一个新的链路; 这里, 所述新的链路可以是原来未发 现的链路, 或是链路的状态发生变化。  Step 305: The controller node A determines, according to the link detection message, that a new link exists between the switch SW1 and another switch SWX. Here, the new link may be a link that is not originally discovered, or a link. The state of the road changes.
步驟 306, 控制器节点 A向全局事件同步系统发布交换机 SW1和交换 机 SWX之间的链路网络事件;  Step 306, the controller node A issues a link network event between the switch SW1 and the switch SWX to the global event synchronization system;
其中, 所述交换机 S W1和交换机 SWX之间的链路网络事件需要携带 链路的基本信息, 如交换机 SW1和交换机 SWX各自的交换机标识、 链路 所在的交换机端口标识以及链路的带宽等状态信息。  The link network event between the switch S W1 and the switch SWX needs to carry basic information about the link, such as the switch identifier of the switch SW1 and the switch SWX, the switch port identifier where the link is located, and the bandwidth of the link. information.
步驟 307, 全局事件同步系统将交换机 SW1和交换机 SWX的链路网 络事件在分布式控制器网络中进行同步, 分布式控制器网络中的其它控制 器节点获知交换机 SW1和交换机 SWX之间的链路发生变化。  Step 307: The global event synchronization system synchronizes the link network events of the switch SW1 and the switch SWX in the distributed controller network, and the other controller nodes in the distributed controller network learn the link between the switch SW1 and the switch SWX. A change has occurred.
步驟 308, 控制器节点 A发现交换机 SW1离开网络。 步驟 309, 控制器节点 A向全局事件同步系统发布交换机 SW1离开的 网络事件; Step 308, controller node A finds that switch SW1 leaves the network. Step 309, the controller node A issues a network event that the switch SW1 leaves to the global event synchronization system;
其中, 所述交换机 SW1 离开的网络事件需要携带交换机 SW1 的基本 信息, 例如交换机的标识等。  The network event that the switch SW1 leaves needs to carry the basic information of the switch SW1, such as the identifier of the switch.
步驟 310, 全局事件同步系统将交换机 SW1 离开的网络事件在分布式 控制器网络中进行同步, 分布式控制器网络中的其它控制器节点获知控制 器节点 A下有交换机 SW1离开网络,并更新各自存储的全局网络拓朴信息。  Step 310: The global event synchronization system synchronizes the network events that the switch SW1 leaves in the distributed controller network, and the other controller nodes in the distributed controller network learn that the switch SW1 leaves the network under the controller node A, and updates each Stored global network topology information.
本实施例中, 利用全局事件同步系统进行网络拓朴变化事件在所有控 制器节点之间的同步, 也可以通过分布式控制器网络的消息广播机制来实 现。  In this embodiment, the synchronization of network topology change events between all controller nodes by using the global event synchronization system can also be implemented by a message broadcast mechanism of the distributed controller network.
图 4是本发明实现步驟 102的具体实现方法的流程示意图, 控制器节 点从交换机上报的数据报文中学习主机位置信息, 并将该主机位置信息存 储到分布式控制器网络中, 如图 4所示, 该方法包括以下步驟:  4 is a schematic flowchart of a specific implementation method of the implementation step 106 of the present invention. The controller node learns the host location information from the data packet reported by the switch, and stores the host location information in the distributed controller network, as shown in FIG. 4 . As shown, the method includes the following steps:
步驟 401 , 主机向自身连接的交换机 SW1发送数据报文。  Step 401: The host sends a data packet to the switch SW1 connected to the host.
步驟 402, 交换机 SW1接收数据报文, 查询本地转发流表后发现没有 对应的表项, 因此向控制器节点 A上报该数据报文。  Step 402: The switch SW1 receives the data packet, and after querying the local forwarding flow table, it finds that there is no corresponding entry, and therefore reports the data packet to the controller node A.
步驟 403 , 控制器节点 A对收到的数据报文的源地址进行检查, 如果 该数据报文的源地址是一个新的主机或对应主机的地址发生变化, 则需要 进行主机位置信息的更新;  Step 403: The controller node A checks the source address of the received data packet. If the source address of the data packet is a new host or the address of the corresponding host changes, the host location information needs to be updated.
这里, 控制器节点 A可以选择将主机位置信息存储在本地。  Here, controller node A may choose to store the host location information locally.
步驟 404, 控制器节点 A将主机位置信息存储到分布式控制器网络的 控制器节点 X上, 具体的存储方式与分布式控制器网络的机制有关, 例如 分布式控制器网络是单跳分布式哈希(One-hop DHT )网络, 则控制器节点 A需要对主机的标识进行哈希运算后再存储到控制器节点。  Step 404, the controller node A stores the host location information on the controller node X of the distributed controller network, and the specific storage manner is related to the mechanism of the distributed controller network, for example, the distributed controller network is a single-hop distributed For a One-hop DHT network, controller node A needs to hash the identity of the host and store it in the controller node.
本实施例中, 需要存储的主机位置信息为连接在交换机上主机的 MAC 地址与交换机的端口标识的对应关系、 或连接在交换机上主机的 IP地址与 交换机的端口标识的对应关系、 或网络域标识与交换机的端口标识的对应 关系; 所述网络域标识包括 VLAN标识、 MPLS标识等。 In this embodiment, the host location information that needs to be stored is the MAC of the host connected to the switch. The correspondence between the address and the port identifier of the switch, or the correspondence between the IP address of the host connected to the switch and the port identifier of the switch, or the correspondence between the network domain identifier and the port identifier of the switch; the network domain identifier includes the VLAN identifier, MPLS logo, etc.
步驟 403和 404中, 需要存储的主机位置信息可以以一种或多种索引 的方式进行存储, 例如以主机的 MAC地址、 和 /或 IP地址、 和 /或网络域标 识为关键字进行存储等。  In steps 403 and 404, the host location information that needs to be stored may be stored in one or more indexes, for example, by using the host's MAC address, and/or IP address, and/or the network domain identifier as a key for storage. .
图 5是本发明实现步驟 103的具体实现方法的流程示意图, 连接在交 换机 SW1的主机 A欲与连接在交换机 SW2的主机 B进行通信, 首个数据 报文被交换机 SW1发往控制器节点 A,控制器节点 A通过分布式控制器网 络查询数据报文的目的地址信息, 并根据全局网络拓朴信息选择并建立合 适的交换路径, 如图 5所示, 该方法包括以下步驟:  5 is a schematic flowchart of a specific implementation method of the implementation step 103 of the present invention. The host A connected to the switch SW1 is to communicate with the host B connected to the switch SW2, and the first data packet is sent by the switch SW1 to the controller node A. The controller node A queries the destination address information of the data packet through the distributed controller network, and selects and establishes an appropriate switching path according to the global network topology information. As shown in FIG. 5, the method includes the following steps:
步驟 501 , 主机 A欲与主机 B进行通信, 向连接的交换机 SW1发送第 一个数据报文。  Step 501: Host A wants to communicate with host B, and sends the first data packet to the connected switch SW1.
步驟 502, 交换机 SW1查询本地的转发流表无匹配项后, 向控制器节 点 A上报该数据报文, 用于向控制器节点 A发起数据流处理请求。  Step 502: After the switch SW1 queries the local forwarding flow table that there is no match, the switch reports the data packet to the controller node A, and is used to initiate a data flow processing request to the controller node A.
步驟 503 , 控制器节点 A向分布式控制器网络中的其它控制器节点查 询该数据报文的目的地址信息;  Step 503: The controller node A queries the other controller nodes in the distributed controller network for the destination address information of the data packet.
这里, 根据数据报文的类型, 如单播、 多播、 广播消息, 数据报文的 目的地址信息可以为一个地址列表。  Here, according to the type of the data message, such as unicast, multicast, and broadcast messages, the destination address information of the data message may be an address list.
其中, 如果主机位置信息的存储采用了本地存储机制, 且交换机 SW1 和交换机 SW2均为控制器节点 A管理的交换机, 则控制器节点 A不需要 向分布式控制器网络中的其它控制器节点查找数据报文的目的地址信息。  Wherein, if the storage of the host location information adopts a local storage mechanism, and the switch SW1 and the switch SW2 are switches managed by the controller node A, the controller node A does not need to search for other controller nodes in the distributed controller network. Destination address information of the data packet.
步驟 504, 控制器节点 A根据数据报文的源和目的的地址信息, 并结 合本地存储的全局网络拓朴信息, 为该数据报文选择交换路径, 本实施例 中的交换路径为交换机 SW1和交换机 SW2。 步驟 505 , 控制器节点 A向交换机 SW1下发该数据报文的转发流表。 步驟 506, 控制器节点 A向交换机 SW2下发该数据报文的转发流表; 这里, 如果交换机 SW2与控制器节点 B直接相连, 本步驟应为: 步驟Step 504: The controller node A selects a switching path for the data packet according to the source and destination address information of the data packet and the global network topology information stored in the local network. The switching path in this embodiment is the switch SW1 and Switch SW2. Step 505: The controller node A sends the forwarding flow table of the data packet to the switch SW1. In step 506, the controller node A sends the forwarding flow table of the data packet to the switch SW2. Here, if the switch SW2 is directly connected to the controller node B, the step is:
506a, 控制器节点 A下发转发流表给控制器节点 B, 由控制器节点 B向交 换机 SW2下发该转发流表。 506a, the controller node A sends the forwarding flow table to the controller node B, and the controller node B delivers the forwarding flow table to the switch SW2.
步驟 507, 交换机 SW1根据收到的转发流表对数据报文进行转发, 这 里是向交换机 SW2进行转发。  Step 507: The switch SW1 forwards the data packet according to the received forwarding flow table, where the switch forwards to the switch SW2.
步驟 508, 交换机 SW2根据收到的转发流表对数据报文进行转发, 这 里是向连接的主机 B进行转发。  Step 508: The switch SW2 forwards the data packet according to the received forwarding flow table, where the forwarding is performed to the connected host B.
为实现上述方法, 本发明还提供一种分布式网络控制装置, 该装置为 控制器节点, 图 6是本发明实现分布式网络控制装置的结构示意图, 如图 6 所示, 该装置包括: 控制模块 61、 事件同步模块 62、 分布式处理模块 63 ; 其中,  To achieve the above method, the present invention further provides a distributed network control device, which is a controller node, and FIG. 6 is a schematic structural diagram of a distributed network control device according to the present invention. As shown in FIG. 6, the device includes: Module 61, event synchronization module 62, distributed processing module 63; wherein
控制模块 61 , 用于接收交换机上报的信息; 还用于接收数据流处理请 求, 并在分布式处理模块查询数据流的目的地址后, 根据本地存储的全局 网络拓 4卜信息为所述数据流建立数据交换路径;  The control module 61 is configured to receive the information reported by the switch, and is further configured to receive the data stream processing request, and after the distributed processing module queries the destination address of the data stream, according to the locally stored global network topology information, the data stream is used as the data stream. Establish a data exchange path;
事件同步模块 62, 用于将上报的信息中的网络拓朴变化信息在所有控 制器节点间同步;  The event synchronization module 62 is configured to synchronize network topology change information in the reported information between all controller nodes;
分布式处理模块 63 , 用于将上报的信息中的主机位置信息保存到分布 式控制网络中; 还用于收到数据流处理请求后, 在所述分布式控制器网络 中查询数据流的目的地址信息。  The distributed processing module 63 is configured to save the host location information in the reported information to the distributed control network, and is further configured to: after receiving the data stream processing request, query the data flow in the distributed controller network. Address information.
该装置还包括:  The device also includes:
信息存储模块 64, 用于保存同步的网络拓朴变化信息、 和 /或收到的主 机位置信息, 并根据所述网络拓朴变化信息得到一致的全局网络拓朴信息。  The information storage module 64 is configured to save the synchronized network topology change information, and/or the received host location information, and obtain consistent global network topology information according to the network topology change information.
其中, 所述事件同步模块 62将所述网络拓朴变化信息在所有控制器节 点间同步为: 利用独立的全局事件同步系统在所有控制器节点间同步所述 网络拓朴变化信息, 或利用分布式控制器网络的消息广播机制在所有控制 器节点间同步所述网络拓朴变化信息。 其中, 所述网络拓朴变化信息为: 交换机的属性、 或交换机加入网络、 或交换机离开网络、 或交换机之间的 链路状态。 The event synchronization module 62 will use the network topology change information in all controller sections. Inter-point synchronization is: synchronizing the network topology change information between all controller nodes by using a separate global event synchronization system, or synchronizing the network topology among all controller nodes by using a message broadcast mechanism of a distributed controller network Change information. The network topology change information is: an attribute of the switch, or a link state of the switch joining the network, or the switch leaving the network, or the switch.
其中, 所述分布式处理模块 63将所述主机位置信息保存到分布式控制 网络中为: 将所述主机位置信息保存到分布式控制器网络中的其他控制器 节点?  The distributed processing module 63 saves the host location information to the distributed control network as: saving the host location information to other controller nodes in the distributed controller network?
其中, 所述主机位置信息为: 连接在交换机上主机的数据链路层位置 和 /或网络层位置。 所述数据链路层位置为连接在交换机上主机的 MAC地 址与交换机的端口标识的对应关系; 所述网络层位置为连接在交换机上主 机的 IP地址与交换机的端口标识的对应关系, 或网络域标识与交换机的端 口标识的^应关系。  The host location information is: a data link layer location and/or a network layer location of a host connected to the switch. The data link layer location is a correspondence between a MAC address of a host connected to the switch and a port identifier of the switch; the network layer location is a correspondence between an IP address of the host connected to the switch and a port identifier of the switch, or a network The relationship between the domain ID and the port ID of the switch.
其中, 所述分布式处理模块 63在分布式控制器网络中查询数据流的目 的地址信息为: 在收到数据流处理请求后, 首先在本节点查询数据流的目 的地址信息, 再在分布式控制网络中的其它控制器节点查询数据流的目的 地址信息。 所述目的地址信息为: 主机 MAC地址和 /或 IP地址对应的交换 机的端口标识, 或网络域标识对应的交换机的端口列表信息。  The destination address information of the distributed processing module 63 for querying the data flow in the distributed controller network is: after receiving the data flow processing request, first querying the destination address information of the data flow in the node, and then distributing The other controller nodes in the control network query the destination address information of the data stream. The destination address information is: a port identifier of a switch corresponding to the host MAC address and/or the IP address, or a port list information of the switch corresponding to the network domain identifier.
以上所述, 仅为本发明的较佳实施例而已, 并非用于限定本发明的保 护范围, 凡在本发明的精神和原则之内所作的任何修改、 等同替换和改进 等, 均应包含在本发明的保护范围之内。  The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included. Within the scope of protection of the present invention.

Claims

权利要求书 Claim
1、 一种分布式网络控制方法, 其特征在于, 多个控制器节点组成分布 式控制器网络, 每个控制器节点管理一个或多个交换机设备; 该方法包括: 控制器节点接收交换机上报的信息 , 将所述信息中的网络拓朴变化信 息在所有控制器节点间同步, 并将所述信息中的主机位置信息保存到分布 式控制网络中;  A distributed network control method, characterized in that a plurality of controller nodes form a distributed controller network, and each controller node manages one or more switch devices; the method includes: the controller node receives the report reported by the switch Information, synchronizing network topology change information in the information between all controller nodes, and saving host location information in the information to a distributed control network;
所述控制器节点收到数据流处理请求时, 在分布式控制器网络中查询 数据流的目的地址信息, 并根据本地存储的全局网络拓朴信息为所述数据 流建立数据交换路径。  When receiving the data stream processing request, the controller node queries the distributed controller network for the destination address information of the data stream, and establishes a data exchange path for the data stream according to the locally stored global network topology information.
2、 根据权利要求 1所述的方法, 其特征在于, 所述将网络拓朴变化信 息在所有控制器节点间同步, 为:  2. The method according to claim 1, wherein the synchronizing network topology change information between all controller nodes is:
所述控制器节点利用独立的全局事件同步系统在所有控制器节点间同 步所述网络拓朴变化信息、 或利用分布式控制器网络的消息广播机制在所 有控制器节点间同步所述网络拓朴变化信息。  The controller node synchronizes the network topology change information among all controller nodes by using an independent global event synchronization system, or synchronizes the network topology among all controller nodes by using a message broadcast mechanism of a distributed controller network Change information.
3、 根据权利要求 1或 2所述的方法, 其特征在于,  3. A method according to claim 1 or 2, characterized in that
该方法还包括: 控制器节点保存同步的网络拓朴变化信息, 并根据所 述网络拓 4卜变化信息得到一致的全局网络拓 4卜信息;  The method further includes: the controller node saves the synchronized network topology change information, and obtains a consistent global network topology information according to the network topology change information;
所述网络拓朴变化信息为: 交换机的属性、 或交换机加入网络、 或交 换机离开网络、 或交换机之间的链路状态。  The network topology change information is: the attributes of the switch, or the link state of the switch joining the network, or the switch leaving the network, or the switch.
4、 根据权利要求 1所述的方法, 其特征在于, 所述将主机位置信息保 存到分布式控制网络中, 为: 所述控制器节点将所述主机位置信息保存到 分布式控制器网络中的除本节点之外的控制器节点中;  The method according to claim 1, wherein the saving the host location information to the distributed control network is: the controller node saves the host location information to a distributed controller network In a controller node other than this node;
相应的, 该方法还包括: 收到交换机上报的信息的控制器节点同时在 本地保存收到的主机位置信息。  Correspondingly, the method further includes: the controller node that receives the information reported by the switch simultaneously saves the received host location information locally.
5、 根据权利要求 1或 4所述的方法, 其特征在于, 所述主机位置信息 为: 连接在交换机上主机的数据链路层位置和 /或网络层位置。 The method according to claim 1 or 4, wherein the host location information To: Connect the data link layer location and/or network layer location of the host on the switch.
6、 根据权利要求 5所述的方法, 其特征在于,  6. The method of claim 5, wherein
所述数据链路层位置为:连接在交换机上主机的 MAC地址与交换机的 端口标识的对应关系;  The data link layer location is: a correspondence between a MAC address of a host connected to the switch and a port identifier of the switch;
所述网络层位置为: 连接在交换机上主机的 IP地址与交换机的端口标 识的对应关系、 或网络域标识与交换机的端口标识的对应关系。  The network layer location is: a correspondence between an IP address of a host connected to the switch and a port identifier of the switch, or a correspondence between a network domain identifier and a port identifier of the switch.
7、 根据权利要求 1所述的方法, 其特征在于, 所述在分布式控制器网 络中查询数据流的目的地址信息, 为:  7. The method according to claim 1, wherein the querying the destination address information of the data stream in the distributed controller network is:
控制器节点在收到数据流处理请求后, 首先在本节点查询数据流的目 的地址信息, 再在分布式控制网络中的除本节点之外的控制器节点查询数 据流的目的地址信息。  After receiving the data stream processing request, the controller node first queries the node for the destination address information of the data stream, and then queries the controller node other than the local node in the distributed control network to query the destination address information of the data stream.
8、 根据权利要求 1或 7所述的方法, 其特征在于, 所述目的地址信息 为: 主机 MAC地址和 /或 IP地址对应的交换机的端口标识、 或网络域标识 对应的交换机的端口列表信息。  The method according to claim 1 or 7, wherein the destination address information is: a port identifier of a switch corresponding to a host MAC address and/or an IP address, or a port list information of a switch corresponding to the network domain identifier. .
9、 一种分布式网络控制装置, 其特征在于, 该装置为控制器节点, 该 控制器节点包括: 控制模块、 事件同步模块、 分布式处理模块; 其中, 所述控制模块, 用于接收交换机上报的信息; 还用于接收数据流处理 请求, 并在分布式处理模块查询数据流的目的地址后, 根据本地存储的全 局网络拓 4卜信息为所述数据流建立数据交换路径;  A distributed network control device, wherein the device is a controller node, the controller node includes: a control module, an event synchronization module, and a distributed processing module; wherein the control module is configured to receive a switch The reported information is further configured to receive the data stream processing request, and after the distributed processing module queries the destination address of the data stream, establish a data exchange path for the data stream according to the locally stored global network topology information;
所述事件同步模块, 用于将交换机上报的信息中的网络拓朴变化信息 在所有控制器节点间同步;  The event synchronization module is configured to synchronize network topology change information in the information reported by the switch between all controller nodes;
所述分布式处理模块, 用于将交换机上报的信息中的主机位置信息保 存到分布式控制网络中; 还用于收到数据流处理请求后, 在所述分布式控 制器网络中查询数据流的目的地址信息。  The distributed processing module is configured to save host location information in the information reported by the switch to the distributed control network; and further, after receiving the data stream processing request, querying the data flow in the distributed controller network Destination address information.
10、 根据权利要求 9所述的装置, 其特征在于, 该装置还包括: 信息存储模块, 用于保存同步的网络拓朴变化信息、 和 /或收到的主机 位置信息, 并根据所述网络拓朴变化信息得到一致的全局网络拓朴信息。 10. The device according to claim 9, wherein the device further comprises: The information storage module is configured to save synchronized network topology change information, and/or received host location information, and obtain consistent global network topology information according to the network topology change information.
PCT/CN2013/071151 2012-02-22 2013-01-30 Distributed network control method and device WO2013123846A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201210040978.1A CN102594689B (en) 2012-02-22 2012-02-22 Distributed network control method and device
CN201210040978.1 2012-02-22

Publications (1)

Publication Number Publication Date
WO2013123846A1 true WO2013123846A1 (en) 2013-08-29

Family

ID=46482905

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2013/071151 WO2013123846A1 (en) 2012-02-22 2013-01-30 Distributed network control method and device

Country Status (2)

Country Link
CN (1) CN102594689B (en)
WO (1) WO2013123846A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115378822A (en) * 2022-08-19 2022-11-22 武汉烽火技术服务有限公司 DDS distributed application simulation method and system

Families Citing this family (56)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103166876B (en) * 2011-12-08 2018-05-04 中兴通讯股份有限公司 Data transmission method for uplink and device between OpenFlow network domains
CN102594689B (en) * 2012-02-22 2015-06-10 中兴通讯股份有限公司 Distributed network control method and device
WO2014019205A1 (en) * 2012-08-02 2014-02-06 华为技术有限公司 Method, apparatus, and system for processing data packet
CN102904813B (en) * 2012-11-05 2016-03-02 华为技术有限公司 A kind of method of message repeating and relevant device
US9729425B2 (en) * 2012-11-29 2017-08-08 Futurewei Technologies, Inc. Transformation and unified control of hybrid networks composed of OpenFlow switches and other programmable switches
CN103856417B (en) * 2012-11-30 2018-01-23 中兴通讯股份有限公司 software defined network message forwarding method and system
CN102984058B (en) * 2012-12-05 2017-04-19 华为技术有限公司 Network communication method based on open stream, controller and exchangers
CN103051539B (en) * 2012-12-14 2015-09-16 中兴通讯股份有限公司 A kind of net control implementation method based on DHT, System and Network controller
US9065768B2 (en) * 2012-12-28 2015-06-23 Futurewei Technologies, Inc. Apparatus for a high performance and highly available multi-controllers in a single SDN/OpenFlow network
CN103906074B (en) * 2012-12-31 2018-01-12 华为技术有限公司 Wireless software defines the method to be communicated in network and its device
FI124398B (en) * 2013-01-04 2014-08-15 Tellabs Oy Method and apparatus for determining implementation of a search table for a network element of software-defined network
CN103974380B (en) * 2013-01-24 2018-05-15 新华三技术有限公司 A kind of method and device of terminal access position keep-alive
CN103973568B (en) * 2013-02-05 2017-03-08 上海贝尔股份有限公司 For the method and apparatus for forwarding SDN stream on the net in MPLS core
CN104022953B (en) * 2013-02-28 2018-02-09 新华三技术有限公司 Message forwarding method and device based on open flows Openflow
CN104022960B (en) * 2013-02-28 2017-05-31 新华三技术有限公司 Method and apparatus based on OpenFlow protocol realizations PVLAN
CN103269278B (en) * 2013-04-19 2016-06-29 江苏省未来网络创新研究院 A kind of terminal unit based on SDN accesses in real time, leaves cognitive method
US20160149779A1 (en) * 2013-05-27 2016-05-26 Rangaprasad Sampath System state message in software defined networking
CN104219145B (en) * 2013-05-31 2017-10-17 英业达科技有限公司 Network system and method for routing
CN104219076B (en) * 2013-06-03 2018-07-24 新华三技术有限公司 A kind of method and device of the fast automatic discovery of the network equipment
CN104253770B (en) * 2013-06-27 2017-07-14 新华三技术有限公司 Realize the method and apparatus of the distributed virtual switch system
CN103414644B (en) * 2013-07-31 2016-12-07 华为技术有限公司 Method that list item issues and controller
CN104426693B (en) * 2013-08-27 2018-02-09 新华三技术有限公司 A kind of method and apparatus that forwarding unit port status management is exchanged in software defined network
CN103501260B (en) * 2013-09-12 2017-03-08 北京创毅讯联科技股份有限公司 The network collocating method of enterprise network and access device
CN108183861B (en) 2013-10-26 2021-09-07 华为技术有限公司 Method for SDN switch to acquire accurate flow entries, SDN switch, controller and system
CN104639450B (en) * 2013-11-11 2018-03-23 中国移动通信集团公司 The architecture system of packet transmission control method, device and software defined network
CN104660507B (en) * 2013-11-22 2018-11-16 华为技术有限公司 The control method and device of forwarding data flow routing
CN103746892B (en) * 2013-11-27 2016-01-13 北京邮电大学 A kind of SDN virtualization platform uplink signaling flow label processing method based on OpenFlow
CN104753695B (en) * 2013-12-25 2018-11-30 上海宽带技术及应用工程研究中心 The discovery of SDN network topological structure and real-time rendering system and method
EP3082305B1 (en) * 2013-12-31 2019-05-22 Huawei Technologies Co., Ltd. Message transmission method, apparatus and communication system
CN104796344B (en) * 2014-01-16 2020-01-14 中兴通讯股份有限公司 Method and system for realizing message forwarding based on SDN, Openflow switch and server
CN103716415B (en) * 2014-01-20 2017-08-22 北京交通大学 A kind of resource perception adaptation method with excellent extensibility
WO2015113290A1 (en) * 2014-01-29 2015-08-06 华为技术有限公司 Downlink data processing method, device and system
CN103888369B (en) * 2014-04-10 2019-02-05 广州市高科通信技术股份有限公司 Ethernet communication method, system and SDN switch
CN104980296B (en) * 2014-04-11 2018-06-19 华为技术有限公司 OpenFlow multi controller systems and its management method
WO2015157947A1 (en) * 2014-04-16 2015-10-22 华为技术有限公司 Software defined network based networking method and device
CN105099646A (en) * 2014-05-14 2015-11-25 中兴通讯股份有限公司 Synchronization link determination method and apparatus
CN104135539B (en) 2014-08-15 2018-03-16 华为技术有限公司 Date storage method, SDN controllers and distributed network storage system
CN105490945A (en) * 2014-09-15 2016-04-13 上海贝尔股份有限公司 Method and device for controlling data transmission in control plane
CN104320278B (en) * 2014-10-31 2017-09-22 新华三技术有限公司 Wide Area Network implementation method and equipment based on software defined network SDN
CN105634959A (en) * 2014-10-31 2016-06-01 杭州华三通信技术有限公司 Method and device for distributing flow table items in software definition network
CN104618475B (en) * 2015-01-28 2018-10-30 清华大学 Horizontal direction communication means and SDN systems for isomery SDN network
CN104579801B (en) * 2015-02-10 2018-01-16 广州市品高软件股份有限公司 A kind of dispatching method of software defined network controller cluster
CN106375218B (en) * 2015-07-23 2019-06-21 华为技术有限公司 A kind of message forwarding method and relevant apparatus
CN105406988B (en) * 2015-10-26 2018-08-21 中国电子科技集团公司第二十八研究所 A kind of software definition is anti-to ruin network establishing method
CN105450447B (en) * 2015-11-17 2018-05-18 上海斐讯数据通信技术有限公司 A kind of network system and network operation method
CN105610615B (en) * 2015-12-29 2018-12-11 国网辽宁省电力有限公司 A kind of power distribution network dispatching method and system based on software defined network
CN106130928B (en) * 2016-06-29 2019-12-03 浪潮集团有限公司 Flow control methods and system under a kind of SDN network
CN107135159B (en) * 2017-03-31 2019-11-22 武汉绿色网络信息服务有限责任公司 The method and system that optimal path determines in a kind of SDN network
CN107257295B (en) * 2017-06-14 2020-04-14 广州市品高软件股份有限公司 Scheduling method of distributed architecture software defined network controller
CN108365979A (en) * 2018-01-31 2018-08-03 深信服科技股份有限公司 Across the controller management method of cluster, SDN controllers and storage medium
CN108737198B (en) * 2018-06-22 2021-08-13 深圳市信锐网科技术有限公司 Method, system and related device for rapidly deploying switch
CN111181766B (en) * 2019-12-10 2022-07-26 上海赛治信息技术有限公司 Redundant FC network system and method for realizing dynamic configuration of switch
CN112260971B (en) * 2020-10-21 2021-11-16 湖南大学 Fault tolerance method and device for network equipment system, computer equipment and storage medium
CN115242805B (en) * 2022-06-11 2024-04-19 北京网聚云联科技有限公司 Distributed MQTT message transmission method, transmission system and storage medium based on gossip
CN115208905A (en) * 2022-08-12 2022-10-18 杭州安恒信息技术股份有限公司 Device information synchronization method, device, system, electronic device and storage medium
CN116667991A (en) * 2023-08-01 2023-08-29 北京国科天迅科技股份有限公司 FC-AE switching network time synchronization method and related equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1808988A (en) * 2005-01-19 2006-07-26 华为技术有限公司 Method of implementing synchronization of topology resource information between bearer networks
US20070286205A1 (en) * 2006-01-31 2007-12-13 Johansen Niels T Node repair in a mesh network
CN101599893A (en) * 2008-06-02 2009-12-09 华为技术有限公司 A kind of minute territory network and obtain the method for branch territory network of network topological diagram
CN102083193A (en) * 2009-11-30 2011-06-01 中兴通讯股份有限公司 Method and system for synchronizing network nodes in time division duplex system
CN102594689A (en) * 2012-02-22 2012-07-18 中兴通讯股份有限公司 Distributed network control method and device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2523402A4 (en) * 2010-01-05 2017-10-18 Nec Corporation Communication system, control apparatus, processing rule setting method, packet transmitting method and program

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1808988A (en) * 2005-01-19 2006-07-26 华为技术有限公司 Method of implementing synchronization of topology resource information between bearer networks
US20070286205A1 (en) * 2006-01-31 2007-12-13 Johansen Niels T Node repair in a mesh network
CN101599893A (en) * 2008-06-02 2009-12-09 华为技术有限公司 A kind of minute territory network and obtain the method for branch territory network of network topological diagram
CN102083193A (en) * 2009-11-30 2011-06-01 中兴通讯股份有限公司 Method and system for synchronizing network nodes in time division duplex system
CN102594689A (en) * 2012-02-22 2012-07-18 中兴通讯股份有限公司 Distributed network control method and device

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115378822A (en) * 2022-08-19 2022-11-22 武汉烽火技术服务有限公司 DDS distributed application simulation method and system
CN115378822B (en) * 2022-08-19 2023-06-06 武汉烽火技术服务有限公司 DDS distributed application simulation method and system

Also Published As

Publication number Publication date
CN102594689B (en) 2015-06-10
CN102594689A (en) 2012-07-18

Similar Documents

Publication Publication Date Title
WO2013123846A1 (en) Distributed network control method and device
US10243841B2 (en) Multicast fast reroute at access devices with controller implemented multicast control plane
US9479432B2 (en) DHT-based control network implementation method and system, and network controller
WO2018006671A1 (en) Message sending method and apparatus, network architecture, and computer storage medium
US9504016B2 (en) Optimized multicast routing in a Clos-like network
EP1722520A1 (en) A method for implementing multicast based on multiple service transmission plate
US8730979B2 (en) Method and system for increasing forwarding efficiency of virtual private LAN service network
WO2019109970A1 (en) Network management method and apparatus, electronic device and storage medium
CN108243123B (en) Broadcast message processing method and device, controller and switch
WO2014125395A1 (en) Multicast support for evpn-spbm based on the mldp signaling protocol
JP2012501583A (en) Improved convergence of multicast traffic
WO2018072732A1 (en) Information processing method and device, and computer storage medium
US9054982B2 (en) Satellite controlling bridge architecture
US20140289193A1 (en) Synchronizing forwarding databases in a network device background
US8514696B2 (en) Multicast tree state replication
CN106790146A (en) The system and its operation method of mobile multicast are realized under a kind of cloud environment based on SDN technologies
US9503272B2 (en) Fast convergence with multicast source mobility
US10742431B2 (en) Centralized database based multicast converging
WO2016131359A1 (en) Multicast line switching method and device for ring-shaped network
US9444643B2 (en) IP multicast over split plane wireless LAN
WO2012146104A1 (en) Method, device and system for updating port information
Li et al. Enhanced robustness of control network for Chinese train control system level-3 (CTCS-3) facilitated by software-defined networking architecture
WO2016086721A1 (en) Method, device and system for transmitting multicast data in trill network
Jin et al. Challenges and advances in using IP multicast for overlay data delivery
Xu et al. A novel efficient sdn based broadcast scheme

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13751425

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13751425

Country of ref document: EP

Kind code of ref document: A1