docker cgroup - [docker cookbook] 读书笔记 2

Control Groups (cgroups) provide resource limitations and accounting for containers. From the Linux Kernel documentation:

Control Groups provide a mechanism for aggregating/partitioning sets
of tasks, and all their future children, into hierarchical groups with
specialized behaviour.

In simple terms, they can be compared to the ulimit shell command or the setrlimit system call. Instead of setting the resource limit to a single process, cgroups allow the limiting of resources to a group of processes.

Control groups are split into different subsystems, such as CPU, CPU sets, memory block I/O, and so on. Each subsystem can be used independently or can be grouped with others. The features that cgroups provide are:

  1. Resource limiting: For example, one cgroup can be bound to specific
    CPUs, so all processes in that group would run off given CPUs only
  2. Prioritization: Some groups may get a larger share of CPUs
  3. Accounting: You can measure the resource usage of different
    subsystems for billing
  4. Control: Freezing and restarting groups

Some of the subsystems that can be managed by cgroups are as follows:

  • blkio: It sets I/O access to and from block devices such as disk,
    SSD, and so on
  • Cpu: It limits access to CPU
  • Cpuacct: It generates CPU resource utilization
  • Cpuset: It assigns the CPUs on a multicore system to tasks in a
  • Devices: It devises access to a set of tasks in a cgroup
  • Freezer: It suspends or resumes tasks in a cgroup
  • Memory: It sets limits on memory use by tasks in a cgroup

There are multiple ways to control work with cgroups. Two of the most popular ones are accessing the cgroup virtual filesystem manually and accessing it with the libcgroup library.

docker namespace

There are different types of namespaces and each one of them isolates applications from each other. They are created using the clone system call. One can also attach to existing namespaces.

  1. The pid namespace allows each container to have its own process

    1. Each pid forms its own process hierarchy. A parent
    2. can see the children namespaces and affect them, but a

    child can neither see the parent namespace nor affect it.

  2. The net namespace allows us to have different network interfaces on
    each container, like port. Each net namespace has its own routing
    table and firewall rules.
  3. ipc namespace sepratate IPC (Inter Process Communication) between
    different container's process;
  4. with mnt namespace, a container can have its own set of mounted
    filesystems and root directories, enhenance chroot.
  5. With uts namespace, we can have different hostnames for each
  6. With user namespace support, we can have users who have a nonzero ID
    on the host but can have a zero ID inside the container.

There are ways to share namespaces between the host and container and container and container.

摘自book: docker cookbook, 第一章 introduction and Installation, 第一节 Introduction

[using docker] 读书笔记 4

1) It’s important to set the USER statement in all your Dockerfiles (or change user within any ENTRYPOINT / CMD scripts). If you don’t do this, your processes will be running as root within the container. As UIDs are the same within a container and on the host, should an attacker manage to break the container, they will have root access to the host machine.

2) 查看container 的CPU, 内存, 网络使用情况
docker stats $(docker inspect -f {{.Name}} $(docker ps -q))

3) cAdvisor aggregates and processes various stats and also makes these available through a REST API, for further processing and storage.

[DevOps for Developers] 读书笔记1

dev & ops 分别对change的态度, 决定了...
devOps 是随着agile实践一步步发展而来, agile 先是在开发team, 后延续到ops team. 它强调的是合作, 沟通.

dev: change code, and want to delivery quickly to production;
ops: no change for production, keep it stable;

devops 概念的萌芽发展历程

  1. Patrick Debois coined the term DevOps in 2009 while organizing the
    DevOpsDays conference in Belgium.
  2. Patrick Debois ran a session called “Agile Operations and
    Infrastructure: How Infra-gile Are You?”4 at the Agile 2008
    conference in Toronto and published a paper with a similar name.
  3. Marcel Wegermann published a e-mail list called “Agile System
  4. John Allspaw gave a presentation called “10+ Deploys per Day: Dev
    and Ops Cooperation”7 at the Velocity 2009 conference in San Jose.
  5. Steven Blank published a book called Four Steps to the Epiphany.
  6. Eric Ries published The Lean Startup9 and others have written on the
    “lean startup” scene.
  7. The 451 Group published the first analyst report on DevOps (titled
    “The Rise of DevOps”10) in September 2010.

DevOps can be examined from the following overlapping perspectives:

  1. Metrics and measurement view: This aspect addresses quality and
    testing and stresses shared incentives.
  2. Process view: This aspect covers congruence and flow to gain fast
    feedback and set up a holistic process.
  3. Technical view: This aspect discusses fast feedback through
    automation, particularly automatic releasing, specification by
    example, and infrastructure as code.

Docker Daemon 监听tcp端口, 远程API 调用

默认情况下, Docker Daemon 监听在本地的 unix:///var/run/docker.sock 上, 只允许本地 root 用户 docker client 连接. 如果要想远程连接, 必须监听 tcp 端口.

Docker Daemon 可以选择在启动的时候, 设置监听在 tcp 端口, IPC socket 监听, 或者2个都监听.

  • 首先看一下Docker Daemon 是不是在监听 (ps -aux | grep docker);
  • 如果已经起来了, 先shutdown (service docker stop);
  • 重新启动, 设置tcp 和 sock 同时监听 (sudo docker -H tcp:// -H unix:///var/run/docker.sock -d &);
    官方关于设置Docker Daemon 的文档:

Screen Shot 2015-07-12 at 12.12.17 PM.png

重新查看进程状态, 可以看到 Docker Deamon 已经起来了. 那么就可以远程http连接了 (查看所有container)

使用 RESTful client, 启动一个container, 这里返回204, 代表没有任何错误
Screen Shot 2015-07-12 at 2.30.33 PM.png

官方关于docker remote API 的文档