跳至主要内容

Python 中的 UUID


python 中的 uuid
代码参考 cpython/Lib/uuid.py
版本 3.7

什么是 UUID

UUID 的全称是 Universal Unique Identifier,中文名是 通用唯一标识符;
wikipedia 上 UUID 的定义是
a 128-bit number used to identify information in computer systems
UUID 需要满足两个条件:
  1. 128 bit
  2. 具有标识别计算机系统的能力

UUID 的规则

格式

  1. 将 16 个 8 位字节表示成 32 个十六进制数
  2. 按照 8-4-4-4-12 的格式加上 4 个连字符 “-‘

    版本

    UUID 应该是下面的这种格式
xxxxxxxx-xxxx-Mxxx-Nxxx-xxxxxxxxxxxx
其中 M 开始的 4 位表示版本,N 开始的 4 位表示变种

Name                        Length(bytes)    Length(hex digits)                       Contents

time_low                         4                             8                  integer giving the low 32 bits of the time
time_mid                         2                             4                  integer giving the middle 16 bits of the time|
time_hi_and_version      2                              4                  4-bit “version” in the most significant bits,                                                                                               followed by the high 12 bits of the time
clock_seq_hi_and           2                             4                  1-3 bit “variant” in the most significant bits,  _resclock_seq_low                                                             followed by the 13-15 bit clock sequence|
node                               6                             12                  the 48-bit node id
Nil UUID
Version 1(date-time and MAC address)
Version 2 (date-time and MAC address, DCE security version)
Versions 3 and 5 (namespace name-based)
Version 4 (random)

UUID 在 python 中的实现

Version2 实际应用的比较少,python 中没有实现;
UUID 的实现就是按照规则将代码写出来,但是 UUID 这个类的初始化与获得机器信息的代码还是可以看看的

class UUID

class UUID:
    """Instances of the UUID class represent UUIDs as specified in RFC 4122.
    UUID objects are immutable, hashable, and usable as dictionary keys.
    Converting a UUID to a string with str() yields something in the form
    '12345678-1234-1234-1234-123456789abc'.  The UUID constructor accepts
    five possible forms: a similar string of hexadecimal digits, or a tuple
    of six integer fields (with 32-bit, 16-bit, 16-bit, 8-bit, 8-bit, and
    48-bit values respectively) as an argument named 'fields', or a string
    of 16 bytes (with all the integer fields in big-endian order) as an
    argument named 'bytes', or a string of 16 bytes (with the first three
    fields in little-endian order) as an argument named 'bytes_le', or a
    single 128-bit integer as an argument named 'int'.

    UUIDs have these read-only attributes:

        bytes       the UUID as a 16-byte string (containing the six
                    integer fields in big-endian byte order)

        bytes_le    the UUID as a 16-byte string (with time_low, time_mid,
                    and time_hi_version in little-endian byte order)

        fields      a tuple of the six integer fields of the UUID,
                    which are also available as six individual attributes
                    and two derived attributes:

            time_low                the first 32 bits of the UUID
            time_mid                the next 16 bits of the UUID
            time_hi_version         the next 16 bits of the UUID
            clock_seq_hi_variant    the next 8 bits of the UUID
            clock_seq_low           the next 8 bits of the UUID
            node                    the last 48 bits of the UUID

            time                    the 60-bit timestamp
            clock_seq               the 14-bit sequence number

        hex         the UUID as a 32-character hexadecimal string

        int         the UUID as a 128-bit integer

        urn         the UUID as a URN as specified in RFC 4122

        variant     the UUID variant (one of the constants RESERVED_NCS,
                    RFC_4122, RESERVED_MICROSOFT, or RESERVED_FUTURE)

        version     the UUID version number (1 through 5, meaningful only
                    when the variant is RFC_4122)

        is_safe     An enum indicating whether the UUID has been generated in
                    a way that is safe for multiprocessing applications, via
                    uuid_generate_time_safe(3).
    """

    def __init__(self, hex=None, bytes=None, bytes_le=None, fields=None,
                       int=None, version=None,
                       *, is_safe=SafeUUID.unknown):
        r"""Create a UUID from either a string of 32 hexadecimal digits,
        a string of 16 bytes as the 'bytes' argument, a string of 16 bytes
        in little-endian order as the 'bytes_le' argument, a tuple of six
        integers (32-bit time_low, 16-bit time_mid, 16-bit time_hi_version,
        8-bit clock_seq_hi_variant, 8-bit clock_seq_low, 48-bit node) as
        the 'fields' argument, or a single 128-bit integer as the 'int'
        argument.  When a string of hex digits is given, curly braces,
        hyphens, and a URN prefix are all optional.  For example, these
        expressions all yield the same UUID:

        UUID('{12345678-1234-5678-1234-567812345678}')
        UUID('12345678123456781234567812345678')
        UUID('urn:uuid:12345678-1234-5678-1234-567812345678')
        UUID(bytes='\x12\x34\x56\x78'*4)
        UUID(bytes_le='\x78\x56\x34\x12\x34\x12\x78\x56' +
                      '\x12\x34\x56\x78\x12\x34\x56\x78')
        UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678))
        UUID(int=0x12345678123456781234567812345678)

        Exactly one of 'hex', 'bytes', 'bytes_le', 'fields', or 'int' must
        be given.  The 'version' argument is optional; if given, the resulting
        UUID will have its variant and version set according to RFC 4122,
        overriding the given 'hex', 'bytes', 'bytes_le', 'fields', or 'int'.

        is_safe is an enum exposed as an attribute on the instance.  It
        indicates whether the UUID has been generated in a way that is safe
        for multiprocessing applications, via uuid_generate_time_safe(3).
        """

        if [hex, bytes, bytes_le, fields, int].count(None) != 4:
            raise TypeError('one of the hex, bytes, bytes_le, fields, '
                            'or int arguments must be given')
        if hex is not None:
            hex = hex.replace('urn:', '').replace('uuid:', '')
            hex = hex.strip('{}').replace('-', '')
            if len(hex) != 32:
                raise ValueError('badly formed hexadecimal UUID string')
            int = int_(hex, 16)
        if bytes_le is not None:
            if len(bytes_le) != 16:
                raise ValueError('bytes_le is not a 16-char string')
            bytes = (bytes_le[4-1::-1] + bytes_le[6-1:4-1:-1] +
                     bytes_le[8-1:6-1:-1] + bytes_le[8:])
        if bytes is not None:
            if len(bytes) != 16:
                raise ValueError('bytes is not a 16-char string')
            assert isinstance(bytes, bytes_), repr(bytes)
            int = int_.from_bytes(bytes, byteorder='big')
        if fields is not None:
            if len(fields) != 6:
                raise ValueError('fields is not a 6-tuple')
            (time_low, time_mid, time_hi_version,
             clock_seq_hi_variant, clock_seq_low, node) = fields
            if not 0 <= time_low < 1<<32:
                raise ValueError('field 1 out of range (need a 32-bit value)')
            if not 0 <= time_mid < 1<<16:
                raise ValueError('field 2 out of range (need a 16-bit value)')
            if not 0 <= time_hi_version < 1<<16:
                raise ValueError('field 3 out of range (need a 16-bit value)')
            if not 0 <= clock_seq_hi_variant < 1<<8:
                raise ValueError('field 4 out of range (need an 8-bit value)')
            if not 0 <= clock_seq_low < 1<<8:
                raise ValueError('field 5 out of range (need an 8-bit value)')
            if not 0 <= node < 1<<48:
                raise ValueError('field 6 out of range (need a 48-bit value)')
            clock_seq = (clock_seq_hi_variant << 8) | clock_seq_low
            int = ((time_low << 96) | (time_mid << 80) |
                   (time_hi_version << 64) | (clock_seq << 48) | node)
        if int is not None:
            if not 0 <= int < 1<<128:
                raise ValueError('int is out of range (need a 128-bit value)')
        if version is not None:
            if not 1 <= version <= 5:
                raise ValueError('illegal version number')
            # Set the variant to RFC 4122.
            int &= ~(0xc000 << 48)
            int |= 0x8000 << 48
            # Set the version number.
            int &= ~(0xf000 << 64)
            int |= version << 76
        self.__dict__['int'] = int
        self.__dict__['is_safe'] = is_safe

    def __eq__(self, other):
        if isinstance(other, UUID):
            return self.int == other.int
        return NotImplemented

    # Q. What's the value of being able to sort UUIDs?
    # A. Use them as keys in a B-Tree or similar mapping.

    def __lt__(self, other):
        if isinstance(other, UUID):
            return self.int < other.int
        return NotImplemented

    def __gt__(self, other):
        if isinstance(other, UUID):
            return self.int > other.int
        return NotImplemented

    def __le__(self, other):
        if isinstance(other, UUID):
            return self.int <= other.int
        return NotImplemented

    def __ge__(self, other):
        if isinstance(other, UUID):
            return self.int >= other.int
        return NotImplemented

    def __hash__(self):
        return hash(self.int)

    def __int__(self):
        return self.int

    def __repr__(self):
        return '%s(%r)' % (self.__class__.__name__, str(self))

    def __setattr__(self, name, value):
        raise TypeError('UUID objects are immutable')

    def __str__(self):
        hex = '%032x' % self.int
        return '%s-%s-%s-%s-%s' % (
            hex[:8], hex[8:12], hex[12:16], hex[16:20], hex[20:])

    @property
    def bytes(self):
        return self.int.to_bytes(16, 'big')

    @property
    def bytes_le(self):
        bytes = self.bytes
        return (bytes[4-1::-1] + bytes[6-1:4-1:-1] + bytes[8-1:6-1:-1] +
                bytes[8:])

    @property
    def fields(self):
        return (self.time_low, self.time_mid, self.time_hi_version,
                self.clock_seq_hi_variant, self.clock_seq_low, self.node)

    @property
    def time_low(self):
        return self.int >> 96

    @property
    def time_mid(self):
        return (self.int >> 80) & 0xffff

    @property
    def time_hi_version(self):
        return (self.int >> 64) & 0xffff

    @property
    def clock_seq_hi_variant(self):
        return (self.int >> 56) & 0xff

    @property
    def clock_seq_low(self):
        return (self.int >> 48) & 0xff

    @property
    def time(self):
        return (((self.time_hi_version & 0x0fff) << 48) |
                (self.time_mid << 32) | self.time_low)

    @property
    def clock_seq(self):
        return (((self.clock_seq_hi_variant & 0x3f) << 8) |
                self.clock_seq_low)

    @property
    def node(self):
        return self.int & 0xffffffffffff

    @property
    def hex(self):
        return '%032x' % self.int

    @property
    def urn(self):
        return 'urn:uuid:' + str(self)

    @property
    def variant(self):
        if not self.int & (0x8000 << 48):
            return RESERVED_NCS
        elif not self.int & (0x4000 << 48):
            return RFC_4122
        elif not self.int & (0x2000 << 48):
            return RESERVED_MICROSOFT
        else:
            return RESERVED_FUTURE

    @property
    def version(self):
        # The version bits are only meaningful for RFC 4122 UUIDs.
        if self.variant == RFC_4122:
            return int((self.int >> 76) & 0xf)

getnode

getnode 区别了 windows 与 unix
windows 的时候会从 windll、netbios 、ipconfig 中获取机器信息
uinx 的时候会从 ifconfig、ip、arp、lanscan、netstat 中获取机器信息
如果都失败了就会生成一个随机的 48 bit 的数字并将第八位至为 1
def getnode(*, getters=None):
    global _node
    if _node is not None:
        return _node

    if sys.platform == 'win32':
        getters = _NODE_GETTERS_WIN32
    else:
        getters = _NODE_GETTERS_UNIX

    for getter in getters + [_random_getnode]:
        try:
            _node = getter()
        except:
            continue
        if (_node is not None) and (0 <= _node < (1 << 48)):
            return _node
    assert False, '_random_getnode() returned invalid value: {}'.format(_node)

_popen

为了执行 ifconfig 等命令
python 中写了一个 _popen 的函数来完成命令的执行
def _popen(command, *args):
    import os, shutil, subprocess
    executable = shutil.which(command)
    if executable is None:
        path = os.pathsep.join(('/sbin', '/usr/sbin'))
        executable = shutil.which(command, path=path)
        if executable is None:
            return None
    env = dict(os.environ)
    env['LC_ALL'] = 'C'
    proc = subprocess.Popen((executable,) + args,
                            stdout=subprocess.PIPE,
                            stderr=subprocess.DEVNULL,
                            env=env)
    return proc

评论

此博客中的热门博文

在 LSF 中使用 docker 运行任务

LSF + Docker Table of Contents1. 环境信息2. 修改配置文件在 lsf 上启用 docker3. 验证4. 部署常见问题5. 部署参考链接 1 环境信息 docker 18.09.5
Kernel Version: 3.10.0-862.11.6.el7.x86_64
lsf 10.1.0.6
OS CentOs 7.6.1810 2 修改配置文件在 lsf 上启用 docker 1.conf/lsf.conf
添加/修改LSF_PROCESS_TRACKING=Y LSF_LINUX_CGROUP_ACCT=Y LSB_RESOURCE_ENFORCE="cpu memory" 2.conf/lsf.shared
添加docker Boolean () () (Docker container) 3.conf/lsf.cluster
添加$your-host-name ! ! 1 3.5 () () (docker) 4./conf/lsbatch/$clustername/configdir/lsb.applications
添加Begin Application NAME = app1 CONTAINER = docker[image(ubuntu:latest) options(--rm --network=host --ipc=host -v /etc/passwd:/etc/passwd -v /etc/group:/etc/group) starter(root)] DESCRIPTION = Test Docker Application Profile 1 End Application 5.badmin reconfig 验证是否可用 3 验证 在非 root 用户下, bsub -app app1 -I cat /etc/lsb-release
DISTRIB_ID=Ubuntu DISTRIB_RELEASE=18.04 DISTRIB_CODENAME=bionic DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS" 4 部署常见问题 1.badmin reconfig 出现 …

k8s 源码阅读 -- eviction

Table of Contents1. 前言2. 资料3. 代码详解3.1. 代码参考3.2. 详细 1 前言 在某些情况下 k8s 会出现 evicted 的 pod, 然而这并不在 pod 的生命周期中.这就是 k8s 的驱逐机制。
当机器的一些资源(内存、磁盘)过小时,为了保证 node 不会受到影响,会将 pod 驱逐至其他的机器上 2 资料 可以在 这里看到相关资料
来看一下代码中,驱逐策略是怎样实现的 3 代码详解 3.1 代码参考kubernetes release-1.10 3.2 详细 pkg/kubelet/apis/kubeletconfig/v1beta/default.go
定义了这几个默认值作为阈值
pkg/kubelet/kubelet.go
kubelte 初始化了 eviction manager
在 runtime 相关模块被加载时,eviction manager 被加载进来
开始了 evict 相关的控制循环
接下来是 evict 真正工作的代码
代码目录是 pkg/kubelet/eviction/
主要看该目录下的两个文件 evictionmanager.go helpers.go
pkg/kubelet/eviction/evictionmanager.go
Start 是 evict manager 的入口
这里是一个死循环
循环中的主要函数是 synchronize 用来清理 pod、同步信息。这个就是今天的主角
先看一下 synchronize 的参数 diskInfoProvider podFunc
diskInfoProvider 是一个接口,用来提供磁盘的信息,作为是否发生驱逐的依据。实际函数在 pkg/kubelet/stats/ 下
synchronize 中仅用到了 HasDedicatedImageFs
podFunc 用来获取一个待检查的 pod 列表,实际函数在 pkg/kubelet/kubeletpods.go
首先检查 imagesfs, 数据从 cadvisor 中获取
获得容器信息和 kubelet 总计状态
summaryProvider 的实际函数在 /pkg/kubelet/server/stats/summary.go
开始监视当前的系统状态
监视这些数据 Node.Memory al…

kubernetes cloud controller manager

kubernetes cloud controller manager Table of Contents1. 什么是 cloud controller manager(ccm)2. 能用 ccm 干什么2.1. 现有的 ccm2.1.1. In Tree2.1.2. Out of Tree3. 如何实现一个 ccm4. ccm 背后的秘密4.1. Out of Tree ccm 如何工作5. 参考链接 本文所有代码基于 1.16.0-alpha.2 commit: bdde11a664 所以引用文档版本为 1.15.0 1 什么是 cloud controller manager(ccm) 在说 ccm 之前要了解一个 ccm 的前身 – cloud provider
cloud provider 是为了 k8s 更加容易在公有云环境下而提出的一个方案
比如在 aws azure 等环境下可以让 k8s 的资源与云厂商的资源进行匹配
具体的演进路线等可以阅读 这篇文章 2 能用 ccm 干什么 在思索 ccm 可以做什么时,要思考一个问题:kubernetes 的核心价值在于哪里?
云主机厂商的本质上是在售卖计算资源与服务,而 k8s 的价值是在于管理与调度容器
正如 k8s 描述的一样: Production-Grade Container Scheduling and Management
k8s 更加关心容器的调度与管理,其他的资源也都是为了容器而服务的
那么有什么资源对于 k8s 来说是可以被替代的?
负载均衡、路由、主机
k8s 不关心主机是实际在东京还是西雅图,也不关心负载均衡具体是如何实现的
它只需要主机上的 kubelet 在正常运行,可以通过负载均衡访问到暴露的服务
而这些恰恰是云厂商最为关心的事情,主机的配置、主机的位置、负载均衡的实现、路由如何到达
这时候再来看 ccm 的接口
LoadBalancer() (LoadBalancer, bool) Instances() (Instances, bool) Zones() (Zones, bool) Clusters() (Clusters, bool) Routes() (Routes, bool) ProviderName() string HasClusterID() bool 这样就…