Total Pageviews

Showing posts with label chef. Show all posts
Showing posts with label chef. Show all posts

Wednesday, 29 June 2022

chef-guard-cookbook


The Chef-Guard cookbook installs and configures Chef-Guard into your environment.

Attributes

Please check http://xanzy.io/projects/chef-guard for all needed details

Usage

This cookbook uses Chef-Vault to store the .pem file content, bookshelf key, and bookshelf secret. The pem file is the private key of the Chef user that Chef-Guard uses to interact with the Chef server and Supermarket. Bookshelf is an internal component of your Chef installation and is used to store uploaded cookbook files.

Please see the following examples for creating the vault items:

knife vault create chef-guard chef.pem -M client -S "name:some-node-search" -A your-chef-username --file chef.pem

The bookshelf keys are created by Chef during install time and are saved in either /etc/chef-server/chef-server-secrets.json (for Open Source Chef) or /etc/opscode/private-chef-secrets.json (for Enterpise Chef and Chef 12) and look something like this in those files:

 "bookshelf": {
    "access_key_id": "xxxxxx",
    "secret_access_key": "yyyyyy"
 }

So just get the keys from one of those files and create the vault like this to get yourself going:

knife vault create chef-guard chef.bookshelf -M client -S "name:some-node-search" -A your-chef-username '{"key":"xxxxxx","secret":"yyyyyy"}'

from https://github.com/xanzy/chef-guard-cookbook

 

Chef-Guard

 Chef-Guard protects your Chef server from untested and uncommitted cookbooks.

http://xanzy.io/

 Build Status GoDoc

Chef-Guard is a feature rich Chef add-on that protects your Chef server from untested and uncommitted (i.e. potentially dangerous) cookbooks by running several validations and checks during the cookbook upload process. In addition Chef-Guard will also monitor, audit, save and email (including a diff with the actual change) all configuration changes and is even capable of validating certain changes before passing them through to Chef.

So installing Chef-Guard onto your Chef server(s) will give you a highly configurable component that enables you to configure and enforce a common workflow for all your colleagues working with Chef.

Technically you can think of Chef-Guard as an extremely smart reverse proxy server written in Go and located/installed right in between Nginx and the Chef Server (see the Installation section for more details). This means that Chef-Guard runs completely server-side and does not require any client-side changes! This gives you the freedom to use whatever tools you like (e.g. knife, berks, the webui) to work with your Chef server and Chef-Guard will make sure all these tools follow the same workflow.

Quickstart

Assuming enough Chef knowledge, it shouldn't take more than 30 minutes to get you started!

  • Read the Chef-Guard documentation explaining and describing what Chef-Guard is and how it works
  • Assuming you already have a running Chef environment, walk through the Chef-Guard prerequisites
  • Your now ready to follow the actual installation which (if you prefer) can be done using a cookbook in just a few minutes

Building

You don't need to build Chef-Guard yourself in order to use it. Pre-built binaries, instructions and a ready to use cookbook can all be found here. If however you would like to contribute to Chef-Guard and/or just feel adventurous and want to build Chef-Guard yourself, please see the contributing documentation to get you started.

Getting Help

Please read the docs first!

  • If you have an issue: report it on the issue tracker
  • If you have a question: visit the #chef-guard channel on irc.freenode.net

from https://github.com/xanzy/chef-guard

 

Monday, 2 April 2018

chef-solo

[edit on GitHub]
chef-solo is a command that executes chef-client in a way that does not require the Chef server in order to converge cookbooks. chef-solo uses chef-client’s Chef local mode, and does not support the following functionality present in chef-client / server configurations:
  • Centralized distribution of cookbooks
  • A centralized API that interacts with and integrates infrastructure components
  • Authentication or authorization
Note
chef-solo can be run as a daemon.

from https://docs.chef.io/chef_solo.html

Thursday, 6 April 2017

A simple Chef cookbook

A simple Chef cookbook to configure my web server.


  • Ensure RVM is installed
  • Ensure Vagrant and VirtualBox is installed
  • vagrant box add precise-server-cloudimg-amd64 PATH-TO-FOLDER-WITH/precise-server-cloudimg-amd64-vagrant-disk1.box
  • vagrant plugin install vagrant-berkshelf
  • vagrant plugin install vagrant-omnibus
  • gem install bundler
  • bundle install
  • cp -r PATH-TO-FOLDER-WITH/Certificates files/default/certificates/
  • vagrant up
  •  
  •  from https://github.com/henrylawson/webserver

Friday, 13 January 2017

Chef server + client self contained vagrant sandbox


Chef Server Sandbox

This is a simple setup to get a single chef-server + 1 client up and running for doing work/testing locally. The server will nat port 4000 local to the server vm. The repo has a .chef/knife.rb setup to work relative to the repo. The cookbooks that run on server will copy the validation and setup a knife client for usage from withing the repo against the server thats built. The client vm will use the validation pem to register against the server. All of it should clean up when vagrant destroy is ran.

Installing

Clone this repo
  git clone git://github.com/spheromak/chef-sandbox.git
Get in that repo
  cd chef-sandbox
Install The Required gems
  bundle install
Vagrant UP!
 bundle exec vagrant up 
This should provision your server and a single client.

Using

You can add more clients to the Vagrant file as-needed.
Rebuild a client
 vagrant destroy client1
This will destroy the client vm and remove the client key and node from the chef server

Next Steps

I started this project cause I wanted to be able to test multi-vm setups in jenkins, and as time permits I will continue to push updates.

from  https://github.com/spheromak/chef-sandbox

Wednesday, 18 May 2016

Chef WSGI Cookbooks


This is a set of cookbooks for configuring a server to deploy and run an arbitrary WSGI application. Most of the cookbooks come from the main Opscode repository or are inspired by such. 
This set of cookbooks was originally collected and developed to test out the use of Vagrant to easily deploy Flask applications to a standard environment. It assumes the following things:
  1. You're developing a standard WSGI application
  2. You want to use uWSGI to run your application
  3. You want to use nginx as a proxy to uWSGI
In addition to the standard Chef deploy resource this setup does the following:
  1. Creates a folder named etc within the shared folder for holding configuration files. Specifically the applicatin's uWSGI configuration.
  2. Creates a folder named logs within the shared folder for holding the nginx and uWSGI logs.
  3. Creates a folder named run to hold the uWSGI socket file.
  4. Installs dependencies into a virtualenv if a file named requirements.txt exists in your application's root folder.

Using with Vagrant

You can quickly deploy your WSGI application to a local, virtual Ubuntu server by setting up a quick recipe for your application. For instance, you would create the following file relative to your project:
Path:
cookbooks/my_wsgi_app/recipes/default.rb
Contents:
application "my_wsgi_app" do
  path "/srv/my_wsgi_app"
  owner "nobody"
  group "nogroup"
  repository "https://github.com/path/to-your-app.git"
  revision "master"

  uwsgi do
    wsgi_file "wsgi.py"
    callable "application"
  end
end
If necessary, modify the values for repositorywsgi_file, and callable to match that of your project.
Then edit your Vagrantfile to look like the following:
Vagrant::Config.run do |config|
  config.vm.define :wsgivm do |wsgi_config|
    wsgi_config.vm.box = "lucid64"
    wsgi_config.vm.box_url = "http://files.vagrantup.com/lucid64.box"
    wsgi_config.vm.forward_port 80, 8080
    wsgi_config.vm.provision :chef_solo do |chef|
      chef.cookbooks_path = "cookbooks"
      chef.add_recipe "apt"
      chef.add_recipe "build-essential"
      chef.add_recipe "runit"
      chef.add_recipe "nginx"
      chef.add_recipe "git"
      chef.add_recipe "python"
      chef.add_recipe "supervisor"
      chef.add_recipe "uwsgi"
      chef.add_recipe "my_wsgi_app"
    end
  end
end
Then from the command line you can run:
$ vagrant up
This should create the server, configure it properly, and deploy the application. You should then be able to view it by going to http://127.0.0.1:8080 in a browser on your local machine.

from https://github.com/mattupstate/wsgi-cookbooks

Friday, 13 May 2016

Chef的基本流程

工具介绍

Chef 和 Chef Solo

Chef 是一个自动化工具,用 Opscode 开发, 使用 Ruby DSL 以一种可重复使用的格式表示配置服务器所需的命令
Chef 经常运行在服务器集群的枢纽,统筹安排其他服务器的配置
可以在单机模式下使用 Chef (Chef Solo)
在这种模式下,我们在本地开发平台上定义服务器的各种角色和设置,然后根据需要手动应用这些设置
如果项目不断扩大也不用担心,使用 Chef Solo 制定的自动化配置大多数都可以在 Chef 中使用

Knife 和 Knife Solo

Knife 是一个命令行工具,为本地开发环境中的 Chef 仓库和远程服务器之间提供交互接口
一般情况下,远程服务器必须是主 Chef 服务器。 不过 Knife Solo 允许在单机模式下使用 Chef, 可以直接和需要配置的服务器交互
Knife Solo 使用介绍

Berkshelf

在 Chef 中,安装各组件所需的命令成为「配方」
Berkshelf 就像是配方的 Bundler
我们使用 Berkshelf 获取配置服务器所需的 Chef 配方(特定的版本)

Chef 术语介绍

  • 配方 (recipe)
    定义安装单个组件(例如 Ruby, mysql-server, Monit 等)所需的命令
  • 食谱 (cookbook)
    相关的配方集合,例如,mysql 食谱中包含 mysql-server 配方和 mysql-client 配方
  • 节点 (node)
    要配置的远程服务器
  • 角色 (role)
    一系列配方的组合,应用在节点上
    例如,Postgres Server 角色可能会包含安装 postgres-server 的配方,安装和设置防火墙的操作, 以及安装合适的服务器监控工具的操作
  • 数据包 (data bag)
    配方所用的元数据,保存为 JSON 格式文件。 例如,要创建的用户列表,以及相应的公钥
  • Chef 仓库 (chef repository)
    一系列节点和角色定义

创建 Chef Solo 仓库

  1. 新建文件夹
    mkdir chef_repo
    cd chef_repo
  2. 安装工具
    1. 初始化 bundler
      bundle init
    2. 在刚生成的 Gemfile 文件中写入如下内容:
      source 'https://rubygems.org'
      
      gem 'knife-solo', '0.3.0'
      gem 'chef', '~> 11.10.0'
      gem 'chef-zero', '1.7.2'
      gem 'berkshelf', '~> 2.0.14'
    3. 然后运行如下命令来安装
      bundle install

初始化仓库

使用 bundle exec 确保使用的 Gemfile 中定义的 gem 版本来执行命令
bundle exec knife solo init .
生成若干文件,结构如下:
chef_repo
├── .chef
│   └── knife.rb
├── .gitignore
├── Berksfile
├── Gemfile
├── Gemfile.lock
├── cookbooks
│   └── .gitkeep
├── data_bags
│   └── .gitkeep
├── environments
│   └── .gitkeep
├── nodes
│   └── .gitkeep
├── roles
│   └── .gitkeep
└── site-cookbooks
    └── .gitkeep
下面来说说各个文件及文件夹的作用

knife.rb

Knife 是个命令行工具,可以和远程服务器上的 Chef 交互
Knife Solo 添加了额外的命令,可以在本地开发设备上直接和要配置的服务器交互 (而不用通过 Chef 中央服务器中转)
knife.rb 中包含针对我们这个仓库的 Knife 设置。 这个文件在使用 Chef 中央服务器时更有用, 并不符合我们使用的单机模式,但却是 Knife 的主要设置(也是唯一的设置文件)
Knife 的全部设置选项可以在 http://docs.opscode.com/config_rb_knife.html 中查看
目前,我们要设置的重要选项如下:
  • cookbook_path
    这个选项是个数组,包含很多相对于仓库根目录的路径, 指定角色中使用的食谱和节点定义的存储位置
  • node_path
    相对仓库根目录的路径,指定节点定义存储的位置
  • role_path
    相对仓库根目录的路径,指定角色定义存储的位置
  • data_bag_path
    相对仓库根目录的路径,指定数据包的存储位置
一般情况下,都可以直接使用默认值

Berksfile

在 Berkfile 中定义 Chef 仓库要使用的食谱,以及各自的版本。 然后执行 berks install 命令安装这些食谱。 如果食谱在 metadata.rb 文件用 depends 定义了依赖食谱,Berkshelf 会负责安装这些依赖食谱
和 Bundler 一样,Berkshelf 也会生成一个 Berksfile.lock 文件,写入食谱及相应的版本号

cookbooks

cookbooks 文件夹用来存放使用 Berkshelf 安装的他人编写的食谱
如想存放自己编写的食谱,应放在 site-cookbooks 文件夹
千万别在 cookbooks 文件夹中存放没有使用 Berkshel 来管理的食谱, 因为,每次运行 knife solo 后,cookbooks 文件夹中的内容都会被抹去重新写入

data_bags

如果食谱中要用到大量数据,就可以将数据存放在这里

environments

对应现实中的工作流,比如说生产环境,准备环境,测试环境,开发环境

nodes

存放节点文件

roles

存放角色文件

site-cookbooks

用来存放自己编写的食谱

食谱介绍

以 redis-tlq 食谱为例:https://github.com/TalkingQuickly/redis-tlq
将其作为自己编写的食谱放到 site-cookbooks 文件夹下
文件结构如下:
redis-tlq
├── attributes
│   └── default.rb
├── metadata.rb
├── recipes
│   └── default.rb
└── templates
    └── default
        ├── redis-server.erb
        └── redis.conf.erb
下面是各个文件及文件夹的说明

metadata.rb

元数据文件中详细说明了这个食谱的作用,编写人,依赖件以及版本号
metadata.rb:
# 食谱的名称
name              "redis-tlq"
maintainer        "Ben Dixon"
maintainer_email  "ben@talkingquickly.co.uk"
description       "Installs redis from rwky's ppa"
# 食谱的版本号
version           "0.0.6"

# 配方的名称
recipe "redis-tlq", "Installs redis"

# 指定这个食谱支持的操作系统
# 这个设置很重要,因为我们这个简单的食谱只支持 Ubuntu, 会用到一些只有 Ubuntu 才支持的命令
supports "ubuntu"
使用 Berkshelf 时,version 和 recipe 这两行是关键的设置, Berkshelf 会查看 metadata.rb 文件,确认要使用的食谱中是否有匹配的配方
还有一个选项 depends 这里没提到,用来指定要依赖的其他食谱

recipes/default.rb

Chef 食谱必须包含一个名为 default 的配方
如果在节点或角色定义中使用到了某个食谱,而没有指定要用哪个配方,就会使用这个默认配方
cookbooks/redis-tlq/recipes/default.rb:
# use redis from ppa rather than the one
# available in the package manager. rwky
# builds the stable version and is kept
# consistently up to date. We need python-software-properties
# for add-apt-repository to work

# Chef 首先会使用操作系统的包管理工具检查是否安装了 python-software-properties 代码包
# 如果没有,那就安装
package 'python-software-properties'

# 告诉 Chef, 其中包含的命令是在 bash 中执行的
# 随后的文本会在配方执行时显示出来,让用户知道配方正在做什么
bash 'adding stable redis ppa' do
  # 以 root 这个用户身份去执行
  user 'root'
  code <<-EOC
    add-apt-repository ppa:chris-lea/redis-server
    apt-get update
  EOC
end

package 'redis-server'

# 上述操作只是安装了 Redis, 我们还要对其进行一些自定义设置,
# 再添加初始化脚本用于启动、停止 Redis 服务器,这就轮到 template 出场了
#
# 这段代码的作用如下:
# 1. 在 redis-tlq/template/default 文件夹中寻找文件 redis.conf.erb
# 2. 解析这个文件中的所有 erb 代码,然后把文件复制到节点上,存放位置为 /etc/redis/redis.conf
# 3. 把这个文件的拥有者设为 root 用户组的 root, 权限为 0644
template "/etc/redis/redis.conf" do
  owner "root"
  group "root"
  mode "0644"
  source "redis.conf.erb"
  notifies :run, "execute[restart-redis]", :immediately
end

# ...

模板文件夹的说明

templates 文件夹中的子文件夹对应的是服务器上所安装的操作系统
如果使用的是 Ubuntu 12.04, 上面的 template 代码块就会在下面的路径中按顺序寻找 redis.conf.erb 文件:
  1. templates/ubuntu-12.04/redis.conf.erb
  2. templates/ubuntu/redis.conf.erb
  3. remplates/default/redis.conf.erb
如果要编写复杂的食谱,照应多种操作系统,这种处理方式的强大就体现出来了
目前,我们仅仅使用 default 文件夹

属性

配方可以使用属性 (attribute) 进行定制。 属性可以在多个地方设定,包括:
  • 节点定义
  • 角色定义
  • 食谱默认值
一般情况下,在食谱中会为属性指定默认值,如果需要,可以覆盖默认值
templates/default/redis.conf.erb 文件中有这么一段代码:
<% unless node[:redis] && node[:redis][:dont_bind] %>
  bind 127.0.0.1
<% end %>
如果在节点中有如下代码:
{
  "redis": {
    "dont_bind": true
  }
}
那么就执行 bind 127.0.0.1 这段代码

食谱属性的默认值

在食谱中可以为属性定义默认值。 这往往也是推荐的做法,大多数用户只要做少许改动就可使用
属性的默认值在 attributes/default.rb 文件中定义:
# whethere to prevent redis from binding to 127.0.0.1
default[:redis][:dont_bind] = false
上面代码的作用是:
会把 false 应用到食谱 node[:redis][:dont_bind],除非默认值被覆盖了
关于属性的更多说明,请阅读 OpsCode 网站上的文档:
http://docs.opscode.com/essentials_cookbook_attribute_files.html

角色介绍

新建角色文件
touch roles/redis-server.json
写入如下内容:
{
  "name": "redis-server",
  "description": "Redis server",
  "default_attributes": {
    "redis": {
      "dont_bind": false
    }
  },
  "run_list": [
    "recipe[redis-tlq]",
    "recipe[monit-tlq]",
    "recipe[monit_configs-tlq::redis-server]"
  ],
  "json_class": "Chef::Role"
}
代码解释如下:
  • name
    指定角色的名称
    当 Chef 在运行列表中见到 rolo[redis-server] 时,就会寻找名为 redis-server 的角色。 寻找的范围是 knife.rb 中 role_path 对应文件夹中所有的 JSON 文件
  • description
    描述这个角色的作用
  • default_attributes
    用来设置默认属性
  • run_list
    是一个数组,Chef 会依次安装执行列出的配方和角色
    角色使用如下格式:
    role[role_name]
    配方使用如下格式:
    recipe[cookbook_name::recipe]
  • “json_class”: “Chef::Role”
    如果使用 JSON 格式,那么必须加上这一行
    如果使用 Ruby 格式,则不用加
因为使用到了 2 个额外的食谱 monit-tlqmonit_configs-tlq
前者用于监控系统的各种参数,如果进程崩溃或者超出了预设的参数,Monit 会重启该进程
后者是针对系统组件的 Monit 设置
所以我们需要在仓库根目录的 Berksfile 文件中加入如下内容:
cookbook 'monit-tlq', github: 'TalkingQuickly/monit-tlq', branch: 'master'
cookbook 'monit_configs-tlq', github: 'TalkingQuickly/monit_configs-tlq', branch: 'master'

配置远程服务器

  1. 把公钥复制到远程服务器
    ssh-copy-id root@yourserverip
  2. 在远程服务器上安装 Chef, 在本地仓库根目录执行如下命令:
    bundle exec knife solo prepare root@yourserverip
    若使用 Vagrant 可以用 vagrant ssh-config 命令查看 ssh 配置,然后执行如下类似命令:
    bundle exec knife solo prepare vagrant@127.0.0.1 -p 2222 -i /Users/mwum/.vagrant.d/insecure_private_key
    安装时间稍长,耐心等待吧
    上述命令的作用如下:
    • 在 nodes 文件夹中新建文件 yourserverip.json
    • 通过 SSH 登入远程服务器
    • 检测操作系统的类型
    • 安装相应的 Chef 版本

可能会遇到的问题

如果不确定是什么问题,可以在安装 Chef 的命令加上 -VV 选项来查看 DEBUG 信息
  • 远程服务器的网络有问题,需要设置代理
    1. 用 ssh 登录远程服务器
    2. 在 ~/.bashrc 中加入如下内容:
      export http_proxy="http://your-proxy.company.org:80"
      export https_proxy="http://your-proxy.company.org:80"
      export HTTP_PROXY="http://your-proxy.company.org:80"
      export HTTPS_PROXY="http://your-proxy.company.org:80"
    3. 退出远程服务器,重新安装 Chef

应用配方

  1. 配置节点文件 nodes/yourserverip.json, 加入如下内容:
    {
      "redis": {
        "dont_bind": false
      },
      "run_list": [
        "role[redis-server]"
      ]
    }
    代码上半段是属性定义,run_list 的作用和角色文件中的一样
  2. 应用配方,在仓库根目录下执行如下命令:
    bundle exec knife solo cook root@yourserverip
    这个命令的作用如下:
    • 根据 Berksfile 文件通过 Berkshelf 下载食谱(相当于此命令 `berks vendor cookbookss/)
    • 通过 SSH 登入远程服务器
    • 把本地的配方文件复制到远程服务器
    • 应用配方,并显示输出结果
    如果要经常重用某个节点定义,可以将其另存为,例如 nodes/side_project_server.json, 然后执行如下命令:
    bundle exec knife solo cook root@yourserverip nodes/side_project_server.json
     

Wednesday, 13 January 2016

chef提供“云管理”,“配置管理”,构建虚拟机等

https://www.chef.io/chef/
http://chef.github.io/
https://github.com/chef/chef
https://github.com/chef/chef-web-docs

Tuesday, 24 June 2014

学习chef

Over the last three months I've been learning how to cook with Chef. Prior to this I have stuck faithful to bash scripts for provisioning servers, and Chef/Puppet/Ansible/etc all claim to make this job a whole lot easier. Having now used Chef extensively, I can for sure say it does not make the job of deploying easier.
My first encounter with Chef was when working in a Vagrant box to deploy a simple Django app and a couple of email related services. The first thing I noticed when reading through the recipe was that each block could easily be written as a single line in a bash script - it seems like writing a lot of extra code because it's simply easier than writing it out in bash (with all it's pitfalls). Nonetheless I ploughed on and after a few weeks was writing my own Chef scripts to deploy other services. Then the real problems began.
One great feature of Chef is the ability to include other peoples recipes, taking away a lot of the hard work. However on multiple occaisions I've included recipes which either 1. no longer exist or worse 2. references packages/other recipes/files which don't exist. Not only is this frustrating to deal with, but trawling through Ruby's awful tracebacks is certainly not enjoyable. Finally is the attribute system, with a nice and simple fifteen(!) levels of attributes, which require some serious studying to learn by heart.
In my experience all of these problems result in one thing: a decrease in developer productivity. We're not fighting with wierd configs & bash scripts, but instead spending days debugging Chef code. I'm 100% certain I could bash-script deploy anything (including groups/multiple servers) quicker than I could with Chef.
Thankfully there are many other options, such as:
  • Bash: still my favoruite - extremely powerful & flexible, great for deploying/installing single machines
  • Fabric: is like an enhanced form of bash using Python
  • Ansible: great for keeping consistent configs across larger groups of nodes
from http://pointlessramblings.com/posts/Learning_to_Chef

Monday, 17 February 2014

chef-一个基于ruby的云基础设施自动化框架

Chef is a systems and cloud infrastructure automation framework that makes it easy to deploy servers and applications to any physical, virtual, or cloud location, no matter the size of the infrastructure. The chef-client relies on abstract definitions (known as cookbooks and recipes) that are written in Ruby and are managed like source code. Each definition describes how a specific part of your infrastructure should be built and managed. The chef-client then applies those definitions to servers and applications, as specified, resulting in a fully automated infrastructure. When a new node is brought online, the only thing the chef-client needs to know is which cookbooks and recipes to apply.
Getting Started
    Chef Overview
    Just Enough Ruby for Chef

from http://docs.opscode.com/chef/
http://docs.opscode.com/open_source/
https://github.com/opscode/chef-docs
https://github.com/chef/chef
https://docs.chef.io/