Skip to content

Instantly share code, notes, and snippets.

@Dounm
Dounm / test_broadcast_object_list.py
Created January 6, 2023 02:09
Show the error when broadcast_object_list() was passed list with different length
# !/usr/bin/env python3
import torch
from torch import distributed as dist
dist.init_process_group(backend="nccl")
torch.cuda.set_device(f"cuda:{dist.get_rank()}")
if dist.get_rank() == 0:
print("I am rank 0")
@Dounm
Dounm / monitor_ib_traffic.py
Last active July 30, 2024 15:40
Monitor Infiniband traffic and caculate bandwidth
# Inspired by https://github.com/vpenso/ganglia-sensors/blob/master/lib/python_modules/infiniband.py#/
import logging
import re
import sys
import json
import time
import subprocess
@Dounm
Dounm / cu
Created March 18, 2021 11:01
Test if cudaMemcpyAsync(H2D, different_streams) is sequential
#include <stdio.h>
#define NUM_STREAMS 4
cudaError_t memcpyUsingStreams (float *fDest,
float *fSrc,
int iBytes,
cudaMemcpyKind eDirection)
{
cudaStream_t *pCuStream = NULL;
int iIndex = 0 ;
@Dounm
Dounm / pre-commit
Created January 4, 2021 10:13
Git Hooks for formatting
#!/usr/bin/env bash
clang_format=clang-format
autopep=autopep8
for FILE in $(git diff --cached --name-only)
do
if [[ $FILE == *.py ]]; then
if ! command -v $autopep &> /dev/null; then
@Dounm
Dounm / test
Created March 7, 2019 09:35
code snippets
// t.h
#include <iostream>
using namespace std;
template <typename T1, typename T2>
class A {
public:
void foo();
};