My Project
programmer's documentation
Functions
cs_all_to_all.c File Reference
#include "cs_defs.h"
#include <assert.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "bft_mem.h"
#include "bft_error.h"
#include "bft_printf.h"
#include "cs_assert.h"
#include "cs_block_dist.h"
#include "cs_crystal_router.h"
#include "cs_log.h"
#include "cs_order.h"
#include "cs_timer.h"
#include "cs_all_to_all.h"
Include dependency graph for cs_all_to_all.c:

Functions

cs_all_to_all_tcs_all_to_all_create (size_t n_elts, int flags, const cs_lnum_t *dest_id, const int dest_rank[], MPI_Comm comm)
 Create an all-to-all distributor based on destination rank. More...
 
cs_all_to_all_tcs_all_to_all_create_from_block (size_t n_elts, int flags, const cs_gnum_t *src_gnum, cs_block_dist_info_t bi, MPI_Comm comm)
 Create an all-to-all distributor for elements whose destination rank is determined from global numbers and block distribution information. More...
 
void cs_all_to_all_destroy (cs_all_to_all_t **d)
 Destroy an all-to-all distributor. More...
 
void cs_all_to_all_transfer_dest_rank (cs_all_to_all_t *d, int **dest_rank)
 Transfer ownership of destination rank to an all-to-all distributor. More...
 
void cs_all_to_all_transfer_dest_id (cs_all_to_all_t *d, cs_lnum_t **dest_id)
 Transfer ownership of destination ids to an all-to-all distributor. More...
 
cs_lnum_t cs_all_to_all_n_elts_dest (cs_all_to_all_t *d)
 Get number of elements associated with all-to-all distributor. More...
 
void * cs_all_to_all_copy_array (cs_all_to_all_t *d, cs_datatype_t datatype, int stride, bool reverse, const void *src_data, void *dest_data)
 Communicate array data using all-to-all distributor. More...
 
cs_lnum_tcs_all_to_all_copy_index (cs_all_to_all_t *d, bool reverse, const cs_lnum_t *src_index, cs_lnum_t *dest_index)
 Communicate local index using all-to-all distributor. More...
 
void * cs_all_to_all_copy_indexed (cs_all_to_all_t *d, cs_datatype_t datatype, bool reverse, const cs_lnum_t *src_index, const void *src_data, const cs_lnum_t *dest_index, void *dest_data)
 Communicate local index using all-to-all distributor. More...
 
int * cs_all_to_all_get_src_rank (cs_all_to_all_t *d)
 Get array of source element ranks associated with an all-to-all distributor. More...
 
cs_all_to_all_type_t cs_all_to_all_get_type (void)
 Get current type of all-to-all distributor algorithm choice. More...
 
void cs_all_to_all_set_type (cs_all_to_all_type_t t)
 Set current type of all-to-all distributor algorithm choice. More...
 
void cs_all_to_all_log_finalize (void)
 Log performance information relative to instrumented all-to-all distribution. More...
 

Detailed Description

All-to-all parallel data exchange.

Function Documentation

◆ cs_all_to_all_copy_array()

void* cs_all_to_all_copy_array ( cs_all_to_all_t d,
cs_datatype_t  datatype,
int  stride,
bool  reverse,
const void *  src_data,
void *  dest_data 
)

Communicate array data using all-to-all distributor.

If a destination buffer is provided, it should be of sufficient size for the number of elements returned by cs_all_to_all_n_elts_dest (multiplied by stride and datatype size).

If no buffer is provided, one is allocated automatically, and transferred to the caller (who is responsible for freeing it when no longer needed).

If used in reverse mode, data is still communicated from src_data to dest_buffer or an internal buffer, but communication direction (i.e. source and destination ranks) are reversed.

This is obviously a collective operation, and all ranks must provide the same datatype, stride, and reverse values.

Parameters
[in,out]dpointer to associated all-to-all distributor
[in]datatypetype of data considered
[in]stridenumber of values per entity (interlaced),
[in]reverseif true, communicate in reverse direction
[in]src_datasource data
[out]dest_datapointer to destination data, or NULL
Returns
pointer to destination data (dest_buffer if non-NULL)

◆ cs_all_to_all_copy_index()

cs_lnum_t* cs_all_to_all_copy_index ( cs_all_to_all_t d,
bool  reverse,
const cs_lnum_t src_index,
cs_lnum_t dest_index 
)

Communicate local index using all-to-all distributor.

If a destination buffer is provided, it should be of sufficient size for the number of elements returned by cs_all_to_all_n_elts_dest.

If no buffer is provided, one is allocated automatically, and transferred to the caller (who is responsible for freeing it when no longer needed).

If used in reverse mode, data is still communicated from src_index to dest_index or an internal buffer, but communication direction (i.e. source and destination ranks) are reversed.

This is obviously a collective operation, and all ranks must provide the same value for the reverse parameter.

Parameters
[in,out]dpointer to associated all-to-all distributor
[in]reverseif true, communicate in reverse direction
[in]src_indexsource index
[out]dest_indexpointer to destination index, or NULL
Returns
pointer to destination data (dest_buffer if non-NULL)

◆ cs_all_to_all_copy_indexed()

void* cs_all_to_all_copy_indexed ( cs_all_to_all_t d,
cs_datatype_t  datatype,
bool  reverse,
const cs_lnum_t src_index,
const void *  src_data,
const cs_lnum_t dest_index,
void *  dest_data 
)

Communicate local index using all-to-all distributor.

If a destination buffer is provided, it should be of sufficient size for the number of elements indicated by dest_index[cs_all_to_all_n_elts_dest(d)];

If no buffer is provided, one is allocated automatically, and transferred to the caller (who is responsible for freeing it when no longer needed).

If used in reverse mode, data is still communicated from src_index to dest_index or an internal buffer, but communication direction (i.e. source and destination ranks) are reversed.

This is obviously a collective operation, and all ranks must provide the same value for the reverse parameter.

Parameters
[in,out]dpointer to associated all-to-all distributor
[in]datatypetype of data considered
[in]reverseif true, communicate in reverse direction
[in]src_indexsource index
[in]src_datasource data
[in]dest_indexdestination index
[out]dest_datapointer to destination data, or NULL
Returns
pointer to destination data (dest_buffer if non-NULL)

◆ cs_all_to_all_create()

cs_all_to_all_t* cs_all_to_all_create ( size_t  n_elts,
int  flags,
const cs_lnum_t dest_id,
const int  dest_rank[],
MPI_Comm  comm 
)

Create an all-to-all distributor based on destination rank.

This is a collective operation on communicator comm.

If the flags bit mask matches CS_ALL_TO_ALL_USE_DEST_ID, data exchanged will be ordered by the array passed to the dest_id argument. For n total values received on a rank (as given by cs_all_to_all_n_elts_dest), those destination ids must be in the range [0, n[.

If the flags bit mask matches CS_ALL_TO_ALL_ORDER_BY_SRC_RANK, data exchanged will be ordered by source rank (this is incompatible with CS_ALL_TO_ALL_USE_DEST_ID.

Attention
The dest_rank and dest_id arrays are only referenced by the distributor, not copied, and must remain available throughout the distributor's lifetime. They may be fully transferred to the structure if not needed elsewhere using the cs_all_to_all_transfer_dest_rank and cs_all_to_all_transfer_dest_id functions.
Parameters
[in]n_eltsnumber of elements
[in]flagssum of ordering and metadata flag constants
[in]dest_idelement destination id (required if flags contain CS_ALL_TO_ALL_USE_DEST_ID), or NULL
[in]dest_rankdestination rank for each element
[in]commassociated MPI communicator
Returns
pointer to new all-to-all distributor

◆ cs_all_to_all_create_from_block()

cs_all_to_all_t* cs_all_to_all_create_from_block ( size_t  n_elts,
int  flags,
const cs_gnum_t src_gnum,
cs_block_dist_info_t  bi,
MPI_Comm  comm 
)

Create an all-to-all distributor for elements whose destination rank is determined from global numbers and block distribution information.

This is a collective operation on communicator comm.

If the flags bit mask matches CS_ALL_TO_ALL_USE_DEST_ID, data exchanged will be ordered by global element number.

If the flags bit mask matches CS_ALL_TO_ALL_ORDER_BY_SRC_RANK, data exchanged will be ordered by source rank (this is incompatible with CS_ALL_TO_ALL_USE_DEST_ID.

Parameters
[in]n_eltsnumber of elements
[in]flagssum of ordering and metadata flag constants
[in]src_gnumglobal source element numbers
[in]bidestination block distribution info
[in]commassociated MPI communicator
Returns
pointer to new all-to-all distributor

◆ cs_all_to_all_destroy()

void cs_all_to_all_destroy ( cs_all_to_all_t **  d)

Destroy an all-to-all distributor.

Parameters
[in,out]dpointer to associated all-to-all distributor

◆ cs_all_to_all_get_src_rank()

int* cs_all_to_all_get_src_rank ( cs_all_to_all_t d)

Get array of source element ranks associated with an all-to-all distributor.

This function should be called only after cs_all_to_all_copy_array, and allocates and returns an array of source element ranks matching the exchanged data elements.

It should also be called only if the distributor creation flags match CS_ALL_TO_ALL_NEED_SRC_RANK or CS_ALL_TO_ALL_ORDER_BY_SRC_RANK.

The returned data is owned by the caller, who is responsible for freeing it when no longer needed.

If source ranks are not available (depending on the distributor creation function and options), the matching pointer will be set to NULL.

Parameters
[in]dpointer to associated all-to-all distributor
Returns
array of source ranks (or NULL)

◆ cs_all_to_all_get_type()

cs_all_to_all_type_t cs_all_to_all_get_type ( void  )

Get current type of all-to-all distributor algorithm choice.

Returns
current type of all-to-all distributor algorithm choice

◆ cs_all_to_all_log_finalize()

void cs_all_to_all_log_finalize ( void  )

Log performance information relative to instrumented all-to-all distribution.

◆ cs_all_to_all_n_elts_dest()

cs_lnum_t cs_all_to_all_n_elts_dest ( cs_all_to_all_t d)

Get number of elements associated with all-to-all distributor.

The number of elements is the number of elements received after exchange.

If no exchange has been done yet (depending on the communication protocol), metadata will be exchanged by this call, so it is a collective operation.

Parameters
[in]dpointer to associated all-to-all distributor
Returns
number of elements associated with distributor.

◆ cs_all_to_all_set_type()

void cs_all_to_all_set_type ( cs_all_to_all_type_t  t)

Set current type of all-to-all distributor algorithm choice.

Parameters
ttype of all-to-all distributor algorithm choice to select

◆ cs_all_to_all_transfer_dest_id()

void cs_all_to_all_transfer_dest_id ( cs_all_to_all_t d,
cs_lnum_t **  dest_id 
)

Transfer ownership of destination ids to an all-to-all distributor.

The dest_id array should be the same as the one used for the creation of the distributor.

Parameters
[in,out]dpointer to associated all-to-all distributor
[in,out]dest_idpointer to element destination id

◆ cs_all_to_all_transfer_dest_rank()

void cs_all_to_all_transfer_dest_rank ( cs_all_to_all_t d,
int **  dest_rank 
)

Transfer ownership of destination rank to an all-to-all distributor.

The dest_rank array should be the same as the one used for the creation of the distributor.

Parameters
[in,out]dpointer to associated all-to-all distributor
[in,out]dest_rankpointer to element destination rank