0%

Lambda 匿名函数

  • 有时候槽函数代码辑逻辑非常简单,可以直接用下面的Lambda匿名函数处理信号,简捷明了.需c++11支持,不支持自身递归调用.

    1
    2
    3
    4
    5
    6
    7
    QComboBox *cb = new QComboBox(this);
    QObject::connect(cb,&QComboBox::currentTextChanged,[=](QString txt){
    [...]
    baseform->changeJsonValue(btn,uname,
    baseform->mWindow->mItemMap.key(txt));
    [...]
    });
  • 函数内部的匿名函数

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38

    void ActionList::onCustomContextMenu(const QPoint &pos)
    {

    [...]
    auto sawp_lambda_func = [this](int src,int dst) {

    QVariantList oldvals;
    for(int i = 0 ; i < mTable->columnCount();i++)
    {
    QWidget *w = mTable->cellWidget(src,i);


    if(!QString::compare(w->metaObject()->className(),"QComboBox"))
    {
    oldvals.append(((QComboBox*)w)->currentText());
    }else{
    oldvals.append(((QLineEdit*)w)->text());
    }
    QObject::disconnect(w);
    mTable->removeCellWidget(src,i);
    w->deleteLater();

    }
    mTable->removeRow(src);
    [...]
    };


    QAction up(QIcon(":/icon/icons/act_up.png") ,
    QString("上移一行"),this);
    QObject::connect(&up,
    &QAction::triggered,[=](){
    sawp_lambda_func(line,line-1);
    });
    [...]
    }

模拟事件

  • 一些GUI的操作,需要一些事件来完成,下面模拟一个鼠标Release的事件

    1
    2
    3
    4
    5
    [...]
    QMouseEvent *event = new QMouseEvent(QMouseEvent::MouseButtonRelease,QCursor::pos(),
    Qt::LeftButton,Qt::LeftButton,Qt::NoModifier);
    QApplication::postEvent(this,event);
    [...]
  • 把事件转换成对其它控件的事件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
void NewGrid::wheelEvent(QWheelEvent *event)
{

if(sliderOrientation == Qt::Horizontal)
{
if(event->orientation() != Qt::Horizontal)
{
QWheelEvent *evt = new QWheelEvent(event->pos(),event->globalPos(),
event->delta(),
event->buttons(),
event->modifiers(),Qt::Horizontal);
QApplication::postEvent(mainScroll->horizontalScrollBar(),evt);
}
}

}

元对像,动态属性

Meta-Object System 的基本功能

  • Meta Object System的设计基于以下几个基础设施:
  • QObject
    • 作为每一个需要利用元对象系统的类的基类
  • QOBJECT
    • 定义在每一个类的私有数据段,用来启用元对象功能,比如,动态属性,信号和槽
  • 元对象编译器moc (the Meta Object Complier)
    • moc分析C++源文件,如果它发现在一个头文件(header file)中包含Q_OBJECT宏定义,然后动态的生成另外一个C++源文件,这个新的源文件包含Q_OBJECT实现代码,这个新的C++源文件也会被编译、链接到这个类的二进制代码中去,因为它也是这个类的完整的一部分.通常,这个新的C++源文件会在以前的C++源文件名前面加上moc作为新文件的文件名.其具体过程如下图所示:
  • 除了提供在对象间进行通讯的机制外,元对象系统还包含以下几种功能:
    • QObject::metaObject()方法
      • 它获得与一个类相关联的meta-object
        -QMetaObject::className()方法
      • 在运行期间返回一个对象的类名,它不需要本地C++编译器的RTTI(run-time type information)支持
    • QObject::inherits()方法
      • 它用来判断生成一个对象类是不是从一个特定的类继承出来,当然,这必须是在QObject类的直接或者间接派生类当中
    • QObject::tr()andQObject::trUtf8()
      • 这两个方法为软件的国际化翻译字符串
    • QObject::setProperty()andQObject::property()
      • 这两个方法根据属性名动态的设置和获取属性值,读写属性
1
2
3
4
5
6
QWidget *w = new QWidget()
w->setProperty("ABC1",4);
w->setProperty("ABC2","dddddd");

qDebug() << w->property("ABC1") << w->property("ABC2");

布署安装

  • 对于开发QT程序来说,最好布署方法就是编译成静态执行文件,只是文件大一点.如果是动态编译就会有一些动态库链接路径的问题.比较特别的它要信赖platforms这个文件路经.可以通过 **QCoreApplication::addLibraryPath(“C:/WINDOWS/System32”);**来指定.

  • 下载安装

  • 参考文档,从样本模版里复制一份出来做相应的修改.

  • 一个安装包的目录结构如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
D:\QtDev\QtInstaller\netmon-root>tree /F
卷 新加卷 的文件夹 PATH 列表
卷序列号码为 0006EE44 E462:E6DB
D:.
│ build.bat

├─config
│ config.xml

└─packages
└─com.vendor.product
├─data
│ │ cares.dll
│ │ libcjson.dll
│ │ libcrypto-1_1.dll
│ │ libcurl.dll
│ │ libeay32_.dll
│ │ libgcc_s_dw2-1.dll
│ │ libmosquitto.dll
│ │ libssl-1_1.dll
│ │ libssp-0.dll
│ │ libstdc++-6.dll
│ │ libwinpthread-1.dll
│ │ libz_.dll
│ │ netmon-tray.exe
│ │ netmonui_i686.dll
│ │ print-mon_i686.dll
│ │ qt.conf
│ │ Qt5Core.dll
│ │ Qt5Gui.dll
│ │ Qt5Network.dll
│ │ Qt5Widgets.dll
│ │ ssleay32_.dll
│ │
│ ├─imageformats
│ │ qdds.dll
│ │ qgif.dll
│ │ qicns.dll
│ │ qico.dll
│ │ qjpeg.dll
│ │ qsvg.dll
│ │ qtga.dll
│ │ qtiff.dll
│ │ qwbmp.dll
│ │ qwebp.dll
│ │
│ └─platforms
│ qminimal.dll
│ qoffscreen.dll
│ qwindows.dll

└─meta
installscript.qs
license.txt
package.xml
page.ui
  • package.xml
1
2
3
4
5
6
7
8
9
10
11
12
<?xml version="1.0" encoding="UTF-8"?>
<Package>
<DisplayName>显示名字</DisplayName>
<Description>随便写一些东西</Description>
<Version>0.1.0-1</Version>
<ReleaseDate>2017-05-11</ReleaseDate>
<Default>script</Default>
<Script>installscript.qs</Script>
<UserInterfaces>
<UserInterface>page.ui</UserInterface>
</UserInterfaces>
</Package>
  • config.xml
1
2
3
4
5
6
7
8
9
<?xml version="1.0" encoding="UTF-8"?>
<Installer>
<Name>程序名字</Name>
<Version>1.0.0</Version>
<Title>程序的抬头</Title>
<Publisher>yjdwbj@gmail.com</Publisher>
<StartMenuDir>网络打印客户端</StartMenuDir>
<TargetDir>@ApplicationsDir@/NetMon</TargetDir>
</Installer>
  • isntallscript.qs
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

function Component()
{
// constructor
component.loaded.connect(this, Component.prototype.loaded);
// 安装完成后自动连接到自动运行的函数
installer.installationFinished.connect(this,Component.prototype.InstallationFinishedAndRun);
installer.finishButtonClicked.connect(this,Component.prototype.installationFinished);
// if (!installer.addWizardPage(component, "Page", QInstaller.TargetDirectory))
// console.log("Could not add the dynamic page.");

}

Component.prototype.createOperations = function()
{
try {
// call the base create operations function
component.createOperations();
component.addOperation("CreateShortcut","C:/WINDOWS/System32/netmon-tray.exe",
"@StartMenuDir@/网络打印机客户端/网络打印机客户端.lnk","@workDirectory=@TargetDir@");
component.addOperation("CreateShortcut","@TargetDir@/maintenancetool.exe",
"@StartMenuDir@/网络打印机客户端/卸载.lnk","@workDirectory=@TargetDir@");

} catch (e) {
console.log(e);
}
}


Component.prototype.createOperationsForArchive = function(archive)
{
# packages\com.vendor.product\data 下面文件全部解压到system32里.
component.addOperation("Extract", archive, "C:/WINDOWS/System32");
}


Version:1.0 StartHTML:0000000107 EndHTML:0000006210 StartFragment:0000000471 EndFragment:0000006172
Component.prototype.InstallationFinishedAndRun = function()
{
try {
if(installer.isInstaller() && installer.status == QInstaller.Success)
{
installer.addWizardPageItem(component,"Page",QInstaller.InstallationFinished);
}
} catch (e) {
console.log(e);
}
}

Component.prototype.installationFinished = function()
{
try {
if(installer.isInstaller() && installer.status == QInstaller.Success)
{
var isRun = component.userInterface("Page").runCheckbox.checked;
if(isRun)
{
QDesktopServices.openUrl("C:/WINDOWS/System32/netmon-tray.exe");
}
}
} catch (e) {
console.log(e);
}
}
  • Page.ui
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
<class>Page</class>
<widget class="QWidget" name="Page">
<property name="geometry">
<rect>
<x>0</x>
<y>0</y>
<width>400</width>
<height>300</height>
</rect>
</property>
<property name="windowTitle">
<string>Dynamic page example</string>
</property>
<layout class="QVBoxLayout" name="verticalLayout">
<item>
<widget class="QLabel" name="m_pageLabel">
<property name="alignment">
<set>Qt::AlignCenter</set>
</property>
</widget>
</item>
<item>
<widget class="QCheckBox" name="runCheckbox">
<property name="text">
<string>安装完成后,运行该程序</string>
</property>
<property name="checked">
<bool>true</bool>
</property>
<property name="tristate">
<bool>false</bool>
</property>
</widget>
</item>
</layout>
</widget>
<resources/>
<connections/>
</ui>
  • 最后这里用一行脚本来生成一键安装包文件.
1
binarycreator --offline-only -c D:\QtDev\QtInstaller\netmon-root\config\config.xml -p D:\QtDev\QtInstaller\netmon-root\packages  "Z:/upload/appname-测试版%DATE%-%TIME::=_%.exe"

调试变量

  • Qt plugin版本不兼容的问题,有时Qt莫名闪退,需设置export QT_DEBUG_PLUGINS=1,方可以看下面的加载.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    Found metadata in lib /fullpath/plugins/sqldrivers/libqsqlite.so, metadata=
    {
    "IID": "org.qt-project.Qt.QSqlDriverFactoryInterface",
    "MetaData": {
    "Keys": [
    "QSQLITE"
    ]
    },
    "archreq": 0,
    "className": "QSQLiteDriverPlugin",
    "debug": false,
    "version": 331264
    }


    Got keys from plugin meta data ("QSQLITE")
    QFactoryLoader::QFactoryLoader() looking at "/fullpath/plugins/sqldrivers/libsqlitecipher.
    Found metadata in lib /fullpath/plugins/sqldrivers/libsqlitecipher.so, metadata=
    {
    "IID": "org.qt-project.Qt.QSqlDriverFactoryInterface",
    "MetaData": {
    "Keys": [
    "SQLITECIPHER"
    ]
    },
    "className": "SqliteCipherDriverPlugin",
    "debug": false,
    "version": 330499
    }
  • 使用export QT_FATAL_WARNINGS=1,可以让程序的在警告的位置崩溃退出,并打印调用stack frame.

  • Could not initialize GLX 错误,尝试设置 export QT_XCB_GL_INTEGRATION=none是可以处理.

  • export USE_WOLFRAM_LD_LIBRARY_PATH=1

1
2
QXcbIntegration: Cannot create platform OpenGL context, neither GLX nor EGL are enabled

  • 关于跨平台的一些全局宏定义如:Q_OS_WIN,Q_OS_MAC,...可以查询参考https://doc.qt.io/qt-5/qtglobal.html

  • 对于工程内的模块,不要加上CONFIG += debug_and_release或者CONFIG += build_all配置,除非是要把它做动态库发布出去的文件,否则这只会增加编译时间,编译了一些必要的对像.

QtCreator调试设置

  • QtCreator的调试插件Using Debugging Helpers,可以参考它,对GDB做一些调脚本.常规选项如,Debugger -> Locals & Expressions -> Display string legnth,默认100字节,可能对于调一些大的json结构体会不够.

Visual Studio上的开发

安装

导入Qt工程

  • msvc也是一个很复杂,很高效的开发工具,如果在windows上调试,还是为它最合适,比竟都是它自家的东西.所以可以通过使用qmake -tp vc -r <your project>.pro的方式,把Qt工程转换成msvc工程,转换后的工程,可能需要少许的手动修改.

  • cl.exe所在位置加入到,系统Path变量里去,用power shell运行qmake -tp vc ../your-project.pro.

  • 或者,通过开始菜单,找到Visual Studio 2019目录,运行x86 Native Tools Command Prompt for VS 2019, 切换到工程目录运行qmake -tp vc.

  • 或者,进入C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build,运行相应的环境Shell.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
     cd "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build"
    PS C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build> ls

    目录: C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build

    Mode LastWriteTime Length Name
    ---- ------------- ------ ----
    d----- 2021/5/11 9:26 14.16
    -a---- 2021/5/11 9:20 13 Microsoft.VCRedistVersion.default.txt
    -a---- 2021/5/11 9:20 393 Microsoft.VCToolsVersion.default.props
    -a---- 2021/5/11 9:20 13 Microsoft.VCToolsVersion.default.txt
    -a---- 2021/5/11 9:26 401 Microsoft.VCToolsVersion.v141.default.props
    -a---- 2021/5/11 9:26 13 Microsoft.VCToolsVersion.v141.default.txt
    -a---- 2021/5/11 9:20 401 Microsoft.VCToolsVersion.v142.default.props
    -a---- 2021/5/11 9:20 13 Microsoft.VCToolsVersion.v142.default.txt
    -a---- 2021/5/11 9:20 39 vcvars32.bat
    -a---- 2021/5/11 9:20 39 vcvars64.bat
    -a---- 2021/5/11 9:20 9859 vcvarsall.bat
    -a---- 2021/5/11 9:20 43 vcvarsamd64_x86.bat
    -a---- 2021/5/11 9:20 43 vcvarsx86_amd64.bat
  • 命令行编译QT,
    Build QT Command Line for Window

    1
    qmake -spec win32-msvc "CONFIG+=qtquickcompiler" ../src/youduqt.pro
  • UTF-8类型添加BOM标识,因为windows下有些程序需要认它.

    1
    2
    printf '\xEF\xBB\xBF' > report_new.csv
    cat report >> report_new.csv
  • 或者这样

    1
    sed -i '1s/^/\xef\xbb\xbf/' file-encoded-with-utf8.txt
  • vs2019奇怪错误,vs2019 C1075, 这个问题在Linux下编译正常,在windows下出错,这是因为这个文件是在Linux下创建,不是UTF8-BOM,所以才出错.这是在不系统下切换开发环境才会碰到的.

调试设置

  • IDE_Debug_Helpers
  • Natvis
  • 把一些*.natvis文件放入到C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\Packages\Debugger\Visualizers里.可以支持调试Qt工程序时,查看对像内的内容,而不是一个指针值.

一些工程总结

  • 工程拆分模块,尽量可以动/静态库的方式链接或者耦合,面向接口编程.
  • 对于可以共用的模块功能,中间的类名,尽量以抽像方式命名,不要带有具体名称,尤其是产品名,公司名.如果要把它移到别的产品就会有点别扭.
  • 对于一些相同功能的类,又没有共同继父类的,类之间尽量不要有全名匹配相同的函数名,当工程到了一规模时,搜索函数名时可以高效过滤找到.
  • 对于大的工程,尽量不要滥用,如:use namespace std;, 对于规模比较大的工程,文件行数多时,在review时无法直观知道它所属的命名空间,而因此碰到命名冲突的概率也很高.
  • 成员变量必须要有一个前缀,如:m_XXXXX,方便有别于局部变量区分.

错误总结

  • Qt5加载QSS资源问题,这个是因为QSS的语法或者配置错误所造成的,会使整个QSS环境坏了.遇到过因为border-image: url(:/x_wnd/home_086@2x.png);而出现下面错误.因为url里的路径里含有@字符,对于路径最好是用双引号(“”)包括起来.而一般的UI工程师切图导出的文件,就是带@符号来区分尺寸的.但是发现在QT C++里写obj->setButtonImage(":/main_wnd/new/02_003_01@2x.png");是可以识别,是正常可行的.
    1
    2
    3
    4
    5
    Could not parse stylesheet of object 0x55dabc30bd60
    Could not parse stylesheet of object 0x55dabc2e8220
    Could not parse stylesheet of object 0x55dabc30bd60
    Could not parse stylesheet of object 0x55dabc30bd60
    Could not parse stylesheet of object 0x55dabc30bd60

QWebChannel使用

谢谢支持

  • 一直以来的服务器开发都会使用PostgreSQL做为数据库.有人会问为什么不用 MySQL?这个是个人喜好.Postgresql 与 MySQL 也是一些区别的.Postgresql 有一些特性是 MySQL 没有具备的.有人会说:在中国为什么 MySQL 的使用会比 PostgreSQL 流行.可能其中一个原因是 PostgreSQL 没有服务好 PHP.PostgreSQL 与 MySQL 的连接方式有很大不同,PostgreSQL 是 fork 进程,MySQL 是线程连接.所以在高并发下 PostgreSQL 的连接数会很快被用完.这时就需要一个连接池处理客户端的连接.
  • Pgbouncer,pgpool,PostgreSQL-XL

安装主机系统

  • 本文的系统是基于Debian 8,安装 qemu-kvm 虚拟机.并于一些相关的软件.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# apt-get install qemu-kvm openvswitch-{switch,common} libvirt-bin virtinst virt-manager
[...]

# /etc/init.d/openvswitch-switch start

[...]


# ovs-vsctl show
504bd042-d300-4fc3-be49-5609ea5df49f
Bridge "ovs-br0"
Port "ovs-br0"
Interface "ovs-br0"
type: internal
ovs_version: "2.5.0"


安装基础系统

  • 本文的做实验的虚拟机系统是基于Centos 7的环境. 通过virsh dumpxml vhost0,配置如下:
  • 两个 CPU,8G 内存,eth0 192.168.120/24, eth1 192.168.25.0/24,  eth0 是桥接主机的 openvswitch 的虚拟网卡用作于 HostOnly 内部网络.eth1 是桥主机的物理网卡,可以连接到互联安装软件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
# virsh dumpxml vhost0
<domain type='kvm' id='53'>
<name>vhost0</name>
<uuid>0fdbcbe7-f7e9-443e-bf9b-7ac4a6d27612</uuid>
<memory unit='KiB'>8388608</memory>
<currentMemory unit='KiB'>8388608</currentMemory>
<vcpu placement='static'>2</vcpu>
<resource>
<partition>/machine</partition>
</resource>
<os>
<type arch='x86_64' machine='pc-i440fx-xenial'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<cpu mode='custom' match='exact'>
<model fallback='allow'>Broadwell-noTSX</model>
</cpu>
<clock offset='utc'>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='pit' tickpolicy='delay'/>
<timer name='hpet' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<pm>
<suspend-to-mem enabled='no'/>
<suspend-to-disk enabled='no'/>
</pm>
<devices>
<emulator>/usr/bin/kvm-spice</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/data/centos7.qcow2'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
</disk>
<controller type='usb' index='0' model='ich9-ehci1'>
<alias name='usb'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x7'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci1'>
<alias name='usb'/>
<master startport='0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0' multifunction='on'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci2'>
<alias name='usb'/>
<master startport='2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x1'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci3'>
<alias name='usb'/>
<master startport='4'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'>
<alias name='pci.0'/>
</controller>
<controller type='ide' index='0'>
<alias name='ide'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
</controller>
<controller type='virtio-serial' index='0'>
<alias name='virtio-serial0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
</controller>
<interface type='direct'>
<mac address='52:54:00:a5:93:d5'/>
<source dev='ovs-system' mode='bridge'/>
<target dev='macvtap10'/>
<model type='virtio'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
<interface type='direct'>
<mac address='52:54:00:14:02:69'/>
<source dev='enp5s0' mode='bridge'/>
<target dev='macvtap11'/>
<model type='virtio'/>
<alias name='net1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
</interface>
<console type='pty' tty='/dev/pts/18'>
<source path='/dev/pts/18'/>
<target type='virtio' port='0'/>
<alias name='console0'/>
</console>
<channel type='unix'>
<source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-pgsql-xl-gmt/org.qemu.guest_agent.0'/>
<target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
<alias name='channel0'/>
<address type='virtio-serial' controller='0' bus='0' port='1'/>
</channel>
<channel type='spicevmc'>
<target type='virtio' name='com.redhat.spice.0' state='disconnected'/>
<alias name='channel1'/>
<address type='virtio-serial' controller='0' bus='0' port='2'/>
</channel>
<input type='tablet' bus='usb'>
<alias name='input0'/>
</input>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<graphics type='spice' port='5905' autoport='yes' listen='127.0.0.1'>
<listen type='address' address='127.0.0.1'/>
</graphics>
<video>
<model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1'/>
<alias name='video0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>
<redirdev bus='usb' type='spicevmc'>
<alias name='redir0'/>
</redirdev>
<redirdev bus='usb' type='spicevmc'>
<alias name='redir1'/>
</redirdev>
<redirdev bus='usb' type='spicevmc'>
<alias name='redir2'/>
</redirdev>
<redirdev bus='usb' type='spicevmc'>
<alias name='redir3'/>
</redirdev>
<memballoon model='virtio'>
<alias name='balloon0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
</memballoon>
</devices>
<seclabel type='dynamic' model='apparmor' relabel='yes'>
<label>libvirt-0fdbcbe7-f7e9-443e-bf9b-7ac4a6d27612</label>
<imagelabel>libvirt-0fdbcbe7-f7e9-443e-bf9b-7ac4a6d27612</imagelabel>
</seclabel>
</domain>

配置基础系统

  • 在虚拟机里安装相关的软件并配置好环境,配置好各虚拟机之间自动使用证书登录 SSH 的功能.
1
2
3
4
5
6
7
8
9
10
11
12
13
# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:a5:93:d5 brd ff:ff:ff:ff:ff:ff
inet 192.168.120.20/24 brd 192.168.120.255 scope global eth0
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:14:02:69 brd ff:ff:ff:ff:ff:ff
inet 192.168.25.20/24 brd 192.168.25.255 scope global eth1
valid_lft forever preferred_lft forever
  • 安装如下的软件包:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

// 安装编译环境
# yum install -y flex bison readline-devel zlib-devel openjade docbook-style-dsssl gcc git tmux libuuid libuuid-devel pcre pcre-devel openssl-devel perl curl

// 添加用户

# groupadd postgres
# useradd -g postgres postgres
# mkdir /home/postgres && chown postgres.postgres -R /home/postgres

// 关闭防火墙,不然会有多问题.
# systemctl disable iptables.service
# iptables -F ; iptables -X

// 关闭SELINUX,修改配置文件/etc/selinux/config,将SELINU置为disabled
# setenforce 0
// OR
# sed -i '/SELINUX/s/enforcing/disabled/' /etc/selinux/config

//预先配置好主机名与IP对应如下

# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.120.19 coord1
192.168.120.18 gtmproxy
192.168.120.20 gtmnode
192.168.120.21 node01
192.168.120.22 node02
192.168.120.23 node03

  • 修改内核参数,系统限制,有些参数要根据需求修改
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
vm.swappiness = 0
net.ipv4.neigh.default.gc_stale_time = 120
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.all.arp_announce = 2
net.ipv4.tcp_max_tw_buckets = 10000
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_max_syn_backlog = 3240000
net.ipv4.tcp_synack_retries = 2
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv4.conf.lo.arp_announce = 2
fs.file-max = 40000500
fs.nr_open = 40000500
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_keepalive_time = 1
net.ipv4.tcp_keepalive_intvl = 4
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_fin_timeout = 5
net.ipv4.tcp_mem = 768432 2097152 15242880
net.ipv4.tcp_rmem = 4096 4096 33554432
net.ipv4.tcp_wmem = 4096 4096 33554432
net.core.somaxconn = 32768
net.ipv4.ip_local_port_range = 1025 65000
net.core.wmem_default = 183888608
net.core.rmem_default = 183888608
net.core.rmem_max = 33554432
net.core.wmem_max = 33554432
net.core.netdev_max_backlog = 2621244
kernel.sem = 250 65536 100 2048
kernel.shmmni = 655360
kernel.shmmax = 34359738368
kernel.msgmni = 65535
kernel.msgmax = 65536
kernel.msgmnb = 65536
net.netfilter.nf_conntrack_max = 1000000
net.nf_conntrack_max = 1000000
kernel.perf_event_max_sample_rate = 6250
net.ipv4.tcp_max_orphans = 1048576
kernel.sched_migration_cost_ns = 5000000
net.core.optmem_max = 25165824
kernel.sem = 10000 2560000 10000 256

# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 515045
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 20000500
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 515045
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited


编译 PostgreSQL-XL

  • 使用 git clone clone 下载源代码postgresql-xl,或者下载源码包.
1
2
3
4
5
6
7
8
9
10
# curl -O http://files.postgres-xl.org/postgres-xl-9.5r1.4.tar.gz
# tar xvf postgres-xl-9.5r1.4.tar.gz
# cd postgres-xl-9.5r1.4 && ./configure && make && make install

[...]

//默认安装在这里
# ls /usr/local/pgsql/
bin include lib share

设置 postgres 用户的变量

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ cat .bashrc
# .bashrc
# User specific aliases and functions

alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'

# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi

export PGUSER=postgres
# export PGHOME=/usr/local/pgxl-9.5-stable
export PGHOME=/usr/local/pgsql
export LD_LIBRARY_PATH=$PGHOME/lib:$LD_LIBRARY_PATH
export PATH=$HOME/bin:$PGHOME/bin:$PATH

配置 PostgreSQL-XL

  • 上面的操作配置了一台虚拟机环境并且编译好了Postgresql-XL,现在此基础上克隆 5 台虚拟机出来,并修它们的网卡与主机名
1
2
// 确保做好SSH公钥认证登录
# for name in node01 node02 node03 coord1 gtmnode gtmproxy; do > ssh ${name} "echo ${name} > /etc/hostname";done
1
2
3
4
5
6
7
8
9
# virsh  list
Id Name State
----------------------------------------------------
52 pgsql-xl-coord1 running
53 pgsql-xl-gmtnode running
54 pgsql-xl-node01 running
55 pgsql-xl-node02 running
56 pgsql-xl-node03 running
57 pgsql-xl-gtmproxy running
  • 所有虚拟机都修改主机名与 IP,并且可以相互自动登录.现在到gtmnode这台机上的postgres用户下去部署 Postgresql-XL 集群了.
  • 使用命令pgxc_ctl prepare config minimal,生成默认配置文件.里面配置比较多.这里要相应的修改.
1
2
3
4
5
 tree /home/postgres/
/home/postgres/
├── pgxc_ctl
│   ├── coordExtraConfig
│   ├── pgxc_ctl.conf
  • 这里根据前面定的虚拟机数量,做了如下修改:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167


[postgres@gtmnode ~]$ cat pgxc_ctl/pgxc_ctl.conf | grep -v '^#' | grep -v '^$' [114/1896]
pgxcInstallDir=$HOME/pgxc
pgxcOwner=$USER # owner of the Postgres-XC databaseo cluster. Here, we use this
# both as linus user and database user. This must be
# the super user of each coordinator and datanode.
pgxcUser=$pgxcOwner # OS user of Postgres-XC owner
tmpDir=/tmp # temporary dir used in XC servers
localTmpDir=$tmpDir # temporary dir used here locally
configBackup=n # If you want config file backup, specify y to this value.
configBackupHost=pgxc-linker # host to backup config file
configBackupDir=$HOME/pgxc # Backup directory
configBackupFile=pgxc_ctl.bak # Backup file name --> Need to synchronize when original changed.
gtmMasterServer=192.168.120.20
gtmMasterPort=20001
gtmMasterDir=$HOME/pgxc/nodes/gtm
gtmExtraConfig=none # Will be added gtm.conf for both Master and Slave (done at initilization only)
gtmMasterSpecificExtraConfig=none # Will be added to Master's gtm.conf (done at initialization only)
gtmSlave=n # Specify y if you configure GTM Slave. Otherwise, GTM slave will not be configured and
# all the following variables will be reset.
gtmSlaveName=gtmSlave
gtmSlaveServer=node12 # value none means GTM slave is not available. Give none if you don't configure GTM Slave.
gtmSlavePort=20001 # Not used if you don't configure GTM slave.
gtmSlaveDir=$HOME/pgxc/nodes/gtm # Not used if you don't configure GTM slave.
gtmSlaveSpecificExtraConfig=none # Will be added to Slave's gtm.conf (done at initialization only)
gtmProxyDir=$HOME/pgxc/nodes/gtm_pxy
gtmProxy=n # Specify y if you conifugre at least one GTM proxy. You may not configure gtm proxies
# only when you dont' configure GTM slaves.
# If you specify this value not to y, the following parameters will be set to default empty values.
# If we find there're no valid Proxy server names (means, every servers are specified
# as none), then gtmProxy value will be set to "n" and all the entries will be set to
# empty values.
gtmProxyNames=(gtm_pxy1 gtm_pxy2 gtm_pxy3 gtm_pxy4) # No used if it is not configured
gtmProxyServers=(node06 node07 node08 node09) # Specify none if you dont' configure it.
gtmProxyPorts=(20001 20001 20001 20001) # Not used if it is not configured.
gtmProxyDirs=($gtmProxyDir $gtmProxyDir $gtmProxyDir $gtmProxyDir) # Not used if it is not configured.
gtmPxyExtraConfig=none # Extra configuration parameter for gtm_proxy. Coordinator section has an example.
gtmPxySpecificExtraConfig=(none none none none)
coordMasterDir=$HOME/pgxc/nodes/coord
coordSlaveDir=$HOME/pgxc/nodes/coord_slave
coordArchLogDir=$HOME/pgxc/nodes/coord_archlog
coordNames=(coord1 coord2 ) # Master and slave use the same name
coordPorts=(20004 20004 ) # Master ports
poolerPorts=(6433 6433 ) # Master pooler ports
coordPgHbaEntries=(192.168.120.0/24) # Assumes that all the coordinator (master/slave) accepts
# the same connection
# This entry allows only $pgxcOwner to connect.
# If you'd like to setup another connection, you should
# supply these entries through files specified below.
coordMasterServers=(coord1 coord2) # none means this master is not available
coordMasterDirs=($coordMasterDir $coordMasterDir )
coordMaxWALsernder=5 # max_wal_senders: needed to configure slave. If zero value is specified,
# it is expected to supply this parameter explicitly by external files
# specified in the following. If you don't configure slaves, leave this value to zero.
coordMaxWALSenders=($coordMaxWALsernder $coordMaxWALsernder )
# max_wal_senders configuration for each coordinator.
# postgresql.conf
cat > $coordExtraConfig <<EOF
log_destination = 'stderr'
logging_collector = on
log_directory = 'pg_log'
listen_addresses = '*'
max_connections = 100
EOF
coordSpecificExtraConfig=(none none none none)
coordExtraPgHba=none # Extra entry for pg_hba.conf. This file will be added to all the coordinators' pg_hba.conf
coordSpecificExtraPgHba=(none none none none)
coordAdditionalSlaves=n # Additional slave can be specified as follows: where you
coordAdditionalSlaveSet=(cad1) # Each specifies set of slaves. This case, two set of slaves are
# configured
cad1_Sync=n # All the slaves at "cad1" are connected with asynchronous mode.
# If not, specify "y"
# The following lines specifies detailed configuration for each
# slave tag, cad1. You can define cad2 similarly.
cad1_Servers=(node08 node09 node06 node07) # Hosts
cad1_dir=$HOME/pgxc/nodes/coord_slave_cad1
cad1_Dirs=($cad1_dir $cad1_dir $cad1_dir $cad1_dir)
cad1_ArchLogDir=$HOME/pgxc/nodes/coord_archlog_cad1
cad1_ArchLogDirs=($cad1_ArchLogDir $cad1_ArchLogDir $cad1_ArchLogDir $cad1_ArchLogDir)
datanodeMasterDir=$HOME/pgxc/nodes/dn_master
datanodeSlaveDir=$HOME/pgxc/nodes/dn_slave
datanodeArchLogDir=$HOME/pgxc/nodes/datanode_archlog
primaryDatanode=node01
datanodeNames=(node01 node02 node03)
datanodePorts=(5432 5432 5432 ) # Master ports
datanodePoolerPorts=(6432 6432 6432 ) # Master pooler ports
datanodePgHbaEntries=(192.168.120.0/24) # Assumes that all the coordinator (master/slave) accepts
# the same connection
# This list sets up pg_hba.conf for $pgxcOwner user.
# If you'd like to setup other entries, supply them
# through extra configuration files specified below.
datanodeMasterServers=(node01 node02 node03) # none means this master is not available.
# This means that there should be the master but is down.
# The cluster is not operational until the master is
# recovered and ready to run.
datanodeMasterDirs=($datanodeMasterDir $datanodeMasterDir $datanodeMasterDir )
datanodeMaxWalSender=5 # max_wal_senders: needed to configure slave. If zero value is
# specified, it is expected this parameter is explicitly supplied
# by external configuration files.
# If you don't configure slaves, leave this value zero.
datanodeMaxWALSenders=($datanodeMaxWalSender $datanodeMaxWalSender $datanodeMaxWalSender )
# max_wal_senders configuration for each datanode
datanodeSlave=n # Specify y if you configure at least one coordiantor slave. Otherwise, the following
# configuration parameters will be set to empty values.
# If no effective server names are found (that is, every servers are specified as none),
# then datanodeSlave value will be set to n and all the following values will be set to
# empty values.
datanodeSlaveServers=(node07 node08 node09 node06) # value none means this slave is not available
datanodeSlavePorts=(20008 20009 20008 20009) # value none means this slave is not available


datanodeSlavePorts=(20008 20009 20008 20009) # value none means this slave is not available
datanodeSlavePoolerPorts=(20012 20013 20012 20013) # value none means this slave is not available
datanodeSlaveSync=y # If datanode slave is connected in synchronized mode
datanodeSlaveDirs=($datanodeSlaveDir $datanodeSlaveDir $datanodeSlaveDir $datanodeSlaveDir)
datanodeArchLogDirs=( $datanodeArchLogDir $datanodeArchLogDir $datanodeArchLogDir $datanodeArchLogDir )
datanodeExtraConfig=none # Extra configuration file for datanodes. This file will be added to all the
# datanodes' postgresql.conf
datanodeSpecificExtraConfig=(none none none none)
datanodeExtraPgHba=none # Extra entry for pg_hba.conf. This file will be added to all the datanodes' postgresql.conf
datanodeSpecificExtraPgHba=(none none none none)
datanodeAdditionalSlaves=n # Additional slave can be specified as follows: where you
# configured
# If not, specify "y"
# The following lines specifies detailed configuration for each
# slave tag, cad1. You can define cad2 similarly.
walArchive=n # If you'd like to configure WAL archive, edit this section.
# Pgxc_ctl assumes that if you configure WAL archive, you configure it
# for all the coordinators and datanodes.
# Default is "no". Please specify "y" here to turn it on.
walArchiveSet=(war1 war2)
war1_source=(master) # you can specify master, slave or ano other additional slaves as a source of WAL archive.
# Default is the master
wal1_source=(slave)
wal1_source=(additiona_coordinator_slave_set additional_datanode_slave_set)
war1_host=node10 # All the nodes are backed up at the same host for a given archive set
war1_backupdir=$HOME/pgxc/backup_war1
wal2_source=(master)
war2_host=node11
war2_backupdir=$HOME/pgxc/backup_war2
coordSlave=n
coordNames=( coord1 none )
coordMasterDirs=( /home/postgres/pgxc/nodes/coord none )
coordPorts=( 20004 -1 )
poolerPorts=( 6433 -1 )
coordMasterServers=( coord1 none )
coordMaxWALSenders=( 5 0 )
coordSlaveServers=( none none )
coordSlavePorts=( none none )
coordSlavePoolerPorts=( none none )
coordSlaveDirs=( none none )
coordArchLogDirs=( none none )
coordSpecificExtraConfig=( none none none none )


### 这里在pgxc_ctl 命令中中修改的,这里的gtmProxy=y 把前的gtmProxy=n给替换掉了,


gtmProxy=y
gtmProxyNames=( gtm_pxy )
gtmProxyServers=( 192.168.120.18 )
gtmProxyPorts=( 20009 )
gtmProxyDirs=( $HOME/pgxc/nodes/gtm_pxy )
gtmPxySpecificExtraConfig=( none )


  • 请注意这里面配置的主机名要与机器一一对应上,关闭防火墙,初始化集群.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[postgres@gtmnode ~]$ pgxc_ctl init all

[...]


[postgres@gtmnode ~]$ pgxc_ctl
/bin/bash
Installing pgxc_ctl_bash script as /home/postgres/pgxc_ctl/pgxc_ctl_bash.
Installing pgxc_ctl_bash script as /home/postgres/pgxc_ctl/pgxc_ctl_bash.
Reading configuration using /home/postgres/pgxc_ctl/pgxc_ctl_bash --home /home/postgres/pgxc_ctl --configuration /home/postgres/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
******** PGXC_CTL START ***************

Current directory: /home/postgres/pgxc_ctl
PGXC help
You are using pgxc_ctl, the configuration utility for PGXL
Type:
help <command>
where <command> is either add, Createdb, Createuser, clean,
configure, deploy, failover, init, kill, log, monitor,
prepare, q, reconnect, remove, set, show, start,
stop or unregister

调优数据库

  • Coordinator 节点的 postgresql.conf
1
2
3
4
5
6
7
8
9
10
11
12
13
14
max_connections = 200          # 允许的最大并发连接数
max_prepared_transactions = 200 # 允许的最大预提交事务数,与更新操作相关
shared_buffers = 8GB # 用于缓存数据的内存(推荐内存的1/4)
temp_buffers = 80MB
work_mem = 100MB # 用于内部排序和一些复杂的查询的内存
dynamic_shared_memory_type = posix
max_files_per_process = 2000
effective_io_concurrency = 8
max_worker_processes = 64
effective_cache_size = 8GB
max_pool_size = 1600 # Coordinator与Datanode之间的连接池的最大连接数
pool_conn_keepalive = 600
persistent_datanode_connections = on

  • Datanode 节点 postgresql.conf
1
2
3
4
5
6
7
8
9
10
11
max_connections = 1600
shared_buffers = 8GB
temp_buffers = 80MB
dynamic_shared_memory_type = posix
effective_io_concurrency = 8
max_worker_processes = 32
random_page_cost = 1.5
effective_cache_size = 24GB
max_prepared_transactions = 100
persistent_datanode_connections = on

  • GtmProxy 节点 gtm_pxy.conf
1
2
3
4
worker_threads = 2
keepalives_idle = 120
keepalives_interval = 30
keepalives_count = 2

Gtm 节点 gtm.conf

1
2
3
keepalives_idle = 120
keepalives_interval = 30
keepalives_count = 50

安装 Pgbouncer 连接池

  • pgbouncer 是 PostgreSQL 的一个轻量级连接池,可以给客户端提供一个统一的视图.

  • pgbouncer 的作用:
    a)pgbouncer 可以在后端数据库和前端应用间简历连接的桥梁,由 pgbouncer 去处理和后端的连接关系.
    b)对客户端的连接进行限制,防止过多的恶意连接.

  • pgbouncer 的特点:
    a)内存消耗低.(默认 2k/连接)
    b)可以把不同的数据库连接到一个机器上,而对客户端保持透明.
    c)支持在线的重行配置而无需重启.


谢谢支持

  • 微信二维码:

  • 做服务器开发绕不开一些测试工作,如功能测试,压力测试,环境的搭建,数据收集,数据分析.这中间有很多重复单调的工作.如果手工一次一次的重复测试,不光费时,还容易出错.这里记录了一些小脚本做为以后工作参考.

提取 TOP 命令中的数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
$ cat grep_pid.sh
#!/bin/bash
srvdir="/root/netty_srv_dir/"

## 分类统计top输出的CPU与内存数据
grep -a "$1 root" $srvdir$1.log | sort -k9 -n | head -n1 | awk '{printf "\nCPU min:%4s\t", $9}'
grep -a "$1 root" $srvdir$1.log | awk '{sum+=$9;n++}END{printf "avg: %4s\t", sum /n }'
grep -a "$1 root" $srvdir$1.log | sort -k9 -n | tail -n1 | awk '{printf " max:%4s\n", $9}'
## print MEM
grep -a "$1 root" $srvdir$1.log | sort -k10 -n | tail -n1 | awk '{printf "MEM: %4s\n",$10}'
cat $srvdir$1.log > $1bak.log
echo "" > $srvdir$1.log
#grep "total" $srvdir$1.net | sort -k 3 |tail -n1 | awk -F";" 'BEGIN{print "Bytes Out\tBytes In\tPackets Out\tPackets In"}{ printf "%sKB/s\t%sKB/s\t%sKB/s\t%sKB/s\n", $3 / 1000,$4/1000,$6/1000,$7/1000}'
#grep "total" $srvdir$1.net | awk 'BEGIN {FS =";";print "BOut\tBin\tPOut\tPin"}{if(NF==16){ if($3>1000){printf "%sKB/s\t",$3/1000}else{printf "%sb/s\t",$3};if($4>1000){printf "%sKB/s\n",$4/1000}else{printf "%sb/s\n",$4}}}' 18028.net
cat /proc/net/sockstat | grep "TCP" | awk '{print $1,$9}'
awk 'BEGIN{print "\nBytesOut\t\tByteIn\t\tPacketsOut\tPacketsIn"}'

## 分类统计bwn-ng输出的CSV格式数据.
grep "total" $srvdir$1.net | sort -t ";" -n -k 3 | tail -n1 | awk -F ";" '{if (NF==16){if($3>1000){printf "%sKB/s\t",$3/1000}else{printf "%sb/s\t\t",$3}}}'
grep "total" $srvdir$1.net | sort -t ";" -n -k 4 | tail -n1 | awk -F ";" '{if (NF==16){if($4>1000){printf "%sKB/s\t",$4/1000}else{printf "%sb/s\t\t",$4}}}'
grep "total" $srvdir$1.net | sort -t ";" -n -k 6 | tail -n1 | awk -F ";" '{if (NF==16){if($6>1000){printf "%sKB/s\t",$6/1000}else{printf "%sb/s\t\t",$6}}}'
grep "total" $srvdir$1.net | sort -t ";" -n -k 7 | tail -n1 | awk -F ";" '{if (NF==16){if($7>1000){printf "%sKB/s\n",$7/1000}else{printf "%sb/s\n",$7}}}'
echo "" > $srvdir$1.net

自动批量回答 yes

  • 这里是一个自动登录的脚本,因为每个服务器第一次连接都会提示要接受一个证书.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
#!/usr/bin/expect -f

set timeout 10

set num [lindex $argv 0]

spawn ssh -b 172.168.$num.10 root@172.168.$num.100
expect {
"*yes/no" { send "yes\r"; exp_continue}
}
expect "#*"
send "exit\r"
send "eof\r"
expect eof

循环测试脚本

  • 这个脚本根据camera_client.py运行打印出``done”表示该程序运行结束,在 while 里把它清除掉.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
root@debian-t6:~# cat test_loop.sh
#!/bin/bash

srv="192.168.25.100"
python camera_client.py dev -H $srv -f $1 &
dpid=$!
echo $dpid
done="done>"
while true;
do
grep "done" camera.log && python camera_client.py app -H $srv -f $1 && kill -9 $dpid && break || sleep 1
done
cat camera.log
echo "" > camera.log

开启服务器与数据收集脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
$ cat test_start_srv.sh
#!/bin/bash

## 设置系统的文件句柄数限制
ulimit -Hn 20000500
ulimit -Sn 20000500
#cd netty_srv_dir

## 清除上次运行的,``bwn-ng, top``命令.
pkill bwm-ng
pkill top
srv="asio_p2p_srv"
## 清除上次运行的服务器的进程
pkill $srv
## 服务已经运行,它的pid赋于给SPID变量,并保存在192.168.25.100机器上的/root/srv.pid里
$srv & SPID=$!
ssh -t root@192.168.25.100 "echo $SPID > srv.pid"

## bwm-ng 监控网卡eth1的流量,并输出CSV格式到以SPID变量名,.net为后缀的文件名里. 它的pid赋于给NPID变量
bwm-ng -o csv -T rate -I eth1 -F $SPID.net & NPID=$!
## top 只输出SPID的pid的信息,并重定向pid.log的文件名中.
top -p $SPID -b > $SPID.log &

服务器测试脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

# cat start_srv.py
import paramiko
import select
ssh = paramiko.SSHClient()
SRV = "192.168.25.100"
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
mykey = paramiko.RSAKey.from_private_key_file("/root/.ssh/id_rsa")
ssh.connect("%s" % SRV,username="root",pkey=mykey)
ssh.exec_command("pkill test_start_srv")
stdin,stdout,stderr = ssh.exec_command("/root/test_start_srv.sh")
while not stdout.channel.exit_status_ready():
if stdout.channel.recv_ready():
rl,wl,xl = select.select([stdout.channel],[],[],0.0)
if len(rl )> 0:
print stdout.channel.recv(1024)
break

1
2
3
4
5
6
7
8
~$ cat test_devices_only.sh
#!/bin/bash

pid=`python start_srv.py`

./test_camera_client.sh $1
ssh -t root@192.168.25.100 "/root/grep_pid.sh $pid && pkill test_start_srv && pkill socket_srv"


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
~$ cat test_app_dev.sh
#!/bin/bash

test_one(){
tt=$200
## 中止远程服务器的之前进程名.
ssh -t root@192.168.25.100 "pkill test_start_srv && pkill socket_srv"
pid=`python start_srv.py`
srv="192.168.25.100"

## 开启一种类型的客户端 ,DEV
python camera_client.py dev -H $srv -f $1 &
dpid=$!
while true;
do
## 等到DEV输出done,开记另一种类型的客户端,APP
grep "done" camera.log && python camera_client.py app -H $srv -f $1 && break || sleep 2
done

## 上面两种类型的客户端运行完成之后,把各类数据分类增量重定向到相应的文件名中.
cat camera.log >> $1.log
echo "" > camera.log
pkill "python"

ssh -t root@192.168.25.100 "pkill test_start_srv && pkill socket_srv && /root/grep_pid.sh $pid" >> $1.log
echo ---------------------------------------------------------------
}


run_ten_test()
{
test_one $1 $2 >> $1.log
echo "$1 test result" >> $1.result
## 分类累计,DEV,APP输出的日志数据,并做平均计算.
grep "DEV Run Time" $1.log | awk '{sum += $2;n++} END {if (n > 0) printf "Dev Run Time: %s\n", sum / n}' >> $1.result
grep "DEV Avg Time" $1.log | awk '{sum += $2;n++} END {if (n > 0) printf "Dev Avg Time: %s\n", sum / n}' >> $1.result
grep "APP Run Time" $1.log | awk '{sum += $2;n++} END {if (n > 0) printf "APP Run Time: %s\n", sum / n}' >> $1.result
grep "APP Avg Time" $1.log | awk '{sum += $2;n++} END {if (n > 0) printf "APP Avg Time: %s\n", sum / n}' >> $1.result

## 分类统计出top 输出的内存与CPU的总数与平均数值
grep "CPU" $1.log | awk '{sum += $2;n++} END {if (n > 0) printf "AVG CPU: %s\n", sum / n}' >> $1.result
grep "MEM" $1.log | awk '{sum += $2;n++} END {if (n > 0) printf "AVG MEM: %s\n", sum / n}' >> $1.result
echo "$1 test result---------------------------------------------------------" >> $1.result
}
test_all()
{
ext="0000.txt"
pre="user_"
for i in `seq 1 6`;
do
echo "start test $pre$i$ext"
test_one $pre$i$ext $i
echo "test $pre$i$ext done"
echo "wait 30 seconds"
sleep 30
done
}
test_all

复制目录,并生成二进制文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

cat ./rsync_openresty.sh
#!/bin/bash

LUAJIT=/usr/local/openresty/luajit/bin/luajit
ORG=openresty-china
BYTECODE=openresty-byte

## 清除旧的目录
rm -rf ${BYTECODE}


## 用find 把目录找出来,替换ORG到BYTECODE.
for i in `find ${ORG} -type d`;do
mkdir -p "${i/"${ORG}"/"${BYTECODE}"}"
done

## 用find 把*.lua结尾的文件找来,并使用LUAJIT编译成字节码,输出到BYTECODE的相应目录下
for i in `find ${ORG} -iname "*.lua"`;do
${LUAJIT} -b $i "${i/"${ORG}"/"${BYTECODE}"}"
done

cp ${ORG}/conf/* ${BYTECODE}/conf

rsync --exclude="*.git" --exclude="*.sh" --exclude="config.lua" --exclude="*.conf" -a ${BYTECODE}/* root@exmaple.com:/home/www/test_openresty

  • 把一个N*M行的转换成NM列如:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
A
A
A
B
B
B
C
C
C

~$ awk '{a[FNR%3] = a[FNR%3] == "" ? $0 : a[FNR%3] "\t" $0} END{for(i=1;i<=3;i++) print a[i%3]}'

A B C
A B C
A B C
  • 或者是这样
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
~$ cat <<EOF | awk -F\| '{a[FNR%4] = a[FNR%4] == "" ? $0 : a[FNR%4] "\t" $0} END{for(i=1;i<=4;i++) print a[i%4]}'
> ADC_CS_N|
> ADC_SADDR|
> ADC_SDAT|
> ADC_SCLK|
> PIN_A10|
> PIN_B10|
> PIN_A9|
> PIN_B14|
> Chip select|
> Digital data input|
> Digital data output|
> Digital clock input|
> 3.3V|
> 3.3V|
> 3.3V|
> 3.3V|
> EOF
ADC_CS_N| PIN_A10| Chip select| 3.3V|
ADC_SADDR| PIN_B10| Digital data input| 3.3V|
ADC_SDAT| PIN_A9| Digital data output| 3.3V|
ADC_SCLK| PIN_B14| Digital clock input| 3.3V|


谢谢支持

  • 微信二维码:

编译Nginx

  • 这里要使用http v2所以 nginx 版要大于1.9.4,nginx 主要用这里的 WEB 前端反向代理.
1
2
# apt-get install libpcre3-dev libssl-dev  zlib1g-dev libzip-dev
# ./configure --with-http_v2_module --with-http_ssl_module --with-threads --with-poll_module --with-file_aio --user=nginx --group=nginx --with-http_gzip_module --with-stream --with-stream_ssl_module --with-pcre

nginx配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
# cat /usr/local/nginx/conf/nginx.conf | grep -v '^$' | grep -v '#'
worker_processes 8;
worker_rlimit_nofile 1048576;
events {
use epoll;
multi_accept on;
worker_connections 65535;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
reset_timedout_connection on;
client_body_timeout 10;
send_timeout 2;
keepalive_timeout 30;
keepalive_requests 500000;
client_body_buffer_size 128k;
client_max_body_size 10m;
client_header_buffer_size 8k;
large_client_header_buffers 16 64k;
output_buffers 1 32k;
postpone_output 1460;
gzip on;
gzip_http_version 1.0;
gzip_proxied any;
gzip_min_length 500;
gzip_disable "MSIE [1-6]\.";
gzip_types text/plain text/xml text/css
text/comma-separated-values
text/javascript
application/x-javascript
application/atom+xml;
server {
listen 443 ssl http2 ;

server_name ssl.example.com;

## 优化HTTPS设置
ssl_session_cache shared:SSL:200m;
ssl_session_timeout 1800m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
add_header Strict-Transport-Security "max-age=31536000" always;
ssl_ciphers AES128-SHA:DES-CBC3-SHA:AES128-SHA256:!ADH:!AECDH:!MD5;

# 这里使用Let's Encrypt 的证书,免费值得信赖. 后面会讲如何申请与更新它.
ssl_certificate /etc/letsencrypt/live/exmaple.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
ssl_trusted_certificate /etc/letsencrypt/live/example.com/fullchain.pem;

access_log off;
location /static/ {
alias /home/www/mqtt_auth/static/;
}

location / {
include uwsgi_params;
# 这里把请求转发给uwsgi 处理.
uwsgi_pass unix:///tmp/uwsgi.sock;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}

申请Let' Encrypt证书

  • Let’s Encrypt 是由互联网安全研究小组(ISRG,一个公益组织)提供的服务.主要赞助商包括电子前哨基金会,Mozilla 基金会,Akamai 以及思科.

  • 这里参照Certbot安装相关的客户端程序.根据自已的域创建一个配置文件,用于自动请求证书.

1
2
3
4
5
6
7
8
9
10
# cat /etc/letsencrypt/configs/www.example.com.conf
domains = www.example.com
rsa-key-size = 2048
email = yjdwbj@gmail.com
text = True

authenticator = standalone

webroot-path = /opt/www


测试NGINX HTTPS设置

  • 查看网站使用的证书
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
openssl s_client -connect  ftp.example.com:18090
[...]
Certificate chain
0 s:/CN=ftp.example.com
i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
....
No client certificate CA names sent
---
SSL handshake has read 2812 bytes and written 631 bytes
---
New, TLSv1/SSLv3, Cipher is AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : AES128-SHA
Session-ID: 47AD316DA0733301D8E2C982BE52168749FF6272CA584466584DA39C28EC2FD9
Session-ID-ctx:
Master-Key: D2EA8F60B4E83D7CA91DD42CA176D6AB21608824C2A7631130291214E7092831B0D795B8E82085E7BCF2D5D32C7A06CA
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 600000 (seconds)
TLS session ticket:
0000 - 3a a1 7f 93 72 a6 d1 04-d9 79 f7 ab 24 0d 17 94 :...r....y..$...
0010 - fa 68 1e e1 1c 6c b0 8f-04 cb ae ad 3a 16 68 b3 .h...l......:.h.
0020 - e4 1a 69 46 30 08 21 50-6f 1d c5 7e d5 77 a8 0c ..iF0.!Po..~.w..
0030 - 66 74 82 f8 3d bb 2f 2a-43 71 65 f3 a9 d0 36 8e ft..=./*Cqe...6.
0040 - 8b 57 d8 a3 4f 9d c8 42-89 7e 9e 27 bc 7d 7a 21 .W..O..B.~.'.}z!
0050 - 5e fe 99 63 b7 20 9d c4-1a ad 80 16 4c cc 67 13 ^..c. ......L.g.
0060 - f1 7e cf 48 f6 ee c4 d6-dc 49 0b f1 c5 58 59 b2 .~.H.....I...XY.
0070 - 2b 4e 74 b6 05 74 b1 3c-68 72 b2 3a 92 3e 31 b9 +Nt..t.<hr.:.>1.
0080 - c6 99 20 7a 80 63 93 e8-e5 7c a5 be 7a 49 a9 43 .. z.c...|..zI.C
0090 - 08 df 34 18 d5 91 0c b0-1d 53 ca b3 24 ea 24 c5 ..4......S..$.$.
00a0 - c0 8f d0 74 76 3d 7d a0-e7 59 fe e7 87 57 68 8f ...tv=}..Y...Wh.

Start Time: 1481641417
Timeout : 300 (sec)
Verify return code: 0 (ok)

SSLScan测试

  • 根据不同的发行版安装 sslcan , apt-get install sslcan.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
$ sslscan -tlsall ftp.example.com:18090
_
___ ___| |___ ___ __ _ _ __
/ __/ __| / __|/ __/ _` | '_ \
\__ \__ \ \__ \ (_| (_| | | | |
|___/___/_|___/\___\__,_|_| |_|

Version 1.8.2
http://www.titania.co.uk
Copyright Ian Ventura-Whiting 2009

Testing SSL server ftp.example.com on port 18090

Supported Server Cipher(s):
Failed SSLv3 256 bits ECDHE-RSA-AES256-GCM-SHA384
Failed SSLv3 256 bits ECDHE-ECDSA-AES256-GCM-SHA384
Failed TLSv1 112 bits SRP-DSS-3DES-EDE-CBC-SHA
Failed TLSv1 112 bits SRP-RSA-3DES-EDE-CBC-SHA
Accepted TLSv1 112 bits DES-CBC3-SHA
Failed TLSv1 112 bits PSK-3DES-EDE-CBC-SHA
Rejected TLSv1 0 bits ECDH-ECDSA-NULL-SHA
Failed TLSv1 0 bits NULL-SHA256
[...]

Prefered Server Cipher(s):
TLSv1 128 bits AES128-SHA

SSL Certificate:
Version: 2
SerialCPS: http://cps.letsencrypt.org
[....]
User Notice:
Explicit Text: This Certificate may only be relied upon by Relying Parties and only in accordance with the Certificate Policy found at https://letsencrypt.org/repository/

Verify Certificate:
unable to get local issuer certificate

NMAP测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
$ nmap --script ssl-enum-ciphers -p 443 ssl.example.com

Starting Nmap 6.47 ( http://nmap.org ) at 2016-12-13 23:13 CST
Nmap scan report for ssl.example.com (18x.2x4.21.5x)
Host is up (0.0063s latency).
PORT STATE SERVICE
443/tcp open https
| ssl-enum-ciphers:
| TLSv1.0:
| ciphers:
| TLS_RSA_WITH_3DES_EDE_CBC_SHA - strong
| TLS_RSA_WITH_AES_128_CBC_SHA - strong
| compressors:
| NULL
| TLSv1.1:
| ciphers:
| TLS_RSA_WITH_3DES_EDE_CBC_SHA - strong
| TLS_RSA_WITH_AES_128_CBC_SHA - strong
| compressors:
| NULL
| TLSv1.2:
| ciphers:
| TLS_RSA_WITH_3DES_EDE_CBC_SHA - strong
| TLS_RSA_WITH_AES_128_CBC_SHA - strong
| TLS_RSA_WITH_AES_128_CBC_SHA256 - strong
| compressors:
| NULL
|_ least strength: strong

Nmap done: 1 IP address (1 host up) scanned in 0.27 seconds

在线检测


Django 环境

  • 安装 virtualenv
1
2
3
4
5
6
7
8
9
10
11

# pip install virtualenv

# cd /opt/www

# virtualenv pyenv

# source pyenv/bin/active
 # 这里通过pip安装项目要用的库.
(pyenv) root@localhost:/opt/www# pip install django uwsgi

Django关键配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
# 使用Redis 保存django的会话.
CACHES = {
"default": {
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION" : "unix://:f4e4821080ca489d3361a520fc2123495755559b45fb24323c5b02e79163e425@/var/run/redis/redis.sock?db=1",
"OPTIONS": {
"DB":0,
"PASSWORD":"f4e4821080ca489d3361a520fc2123495755559b45fb24323c5b02e79163e425",
"CONNECTION_POOL_KWARGS": {"max_connections": 65535},
}
}
}

SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "default"
PASSWORD_HASER=[
"django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher",
"django.contrib.auth.hashers.SHA1PasswordHasher",
]
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'user_manager',
]
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
ROOT_URLCONF = 'my_app.urls'
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [os.path.join(BASE_DIR),'templates'],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
WSGI_APPLICATION = 'my_app.wsgi.application'

# 配置连接接两个数据库
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME' : 'testapp',
'USER' :'testapp',
'PASSWORD':'testapp123',
'HOST' : '127.0.0.1',
## 端口这里可以指定数据库的端口,也可以是连接池的端口
'PORT' : '5432',
},
'seconddb':{
'ENGINE':'django.db.backends.postgresql_psycopg2',
'NAME' : 'otherdb',
'USER' : 'testapp',
'PASSWORD':'testapp123',
'HOST':'127.0.0.1',
'PORT':'5432',
}
}
AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
AUTO_LOGOUT_DELAY = 1
APPEND_SLASH=True
STATICFILES_DIRS = (
os.path.join(BASE_DIR,'static'),
)
STATIC_ROOT = os.path.join(BASE_DIR)
STATIC_URL = '/static/'

uWSGI 配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

project = my_app
base = /home/www
chdir = %(base)/%(project)
module = %(project).wsgi:application
uid = nginx
gid = nginx
listen = 65536
master = true
virtualenv = /home/www/pyenv
home = /home/www/pyenv
master = true
cpu = affinity
async = true
py-autoreload=true
processes = 88
die-on-term = true
#threads = 32
max-requests = 65536
#socket = /tmp/uwsgi.sock
#http = 127.0.0.1:9090
#socket = 192.168.25.109:4599
disable-logging = true
buffer-size =32768
harakiri = 20
chmod-socket = 664
daemonize=/home/www/log/uwsgi.log
vcauum = true

Postgresql配置连接池

  • 如果不为 Postgresl 配置连接池,程序会在高并发的情况下把数据库的连接全部用完
  • 参考链接pgbouncer
  • 下面设置很简单.更多高级功能后面挖掘.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# grep -v "^;" /etc/pgbouncer/pgbouncer.ini  | grep -v "^$"
[databases]
## 下面这个连接信息必需与postgresql里设一样.就是说,postgresql 要有test_db数库名,test用户,它的密码是test123,....
test_db = host=127.0.0.1 port=5432 user=test password=test123 client_encoding=UNICODE datestyle=ISO connect_query='SELECT 1'

[pgbouncer]
pidfile = /var/run/postgresql/pgbouncer.pid
## 绑定所有地址
listen_addr = *
listen_port = 6432
## 通过unix socket与postgresql 通信
unix_socket_dir = /var/run/postgresql
auth_type = md5

## 认证用户的信息,每行格式: "test" "test123"
auth_file = /etc/pgbouncer/userlist.txt
admin_users = admin
pool_mode = transaction
server_reset_query = DISCARD ALL

## 我测试到的一个比较理想的数值,过大过小对程序的并发都有影响.
max_client_conn = 100
default_pool_size = 20
listen_backlog = 1024

总结

  • 在网上看到好多人声称uWSGINginx会达到很高的性能,我用这种配置组合,压测的数据很难看.TPS 1K/sec.安装了连接池之后,虽然没有做读写分享,但是它的提升还是很大的.

谢谢支持

  • 参考链接:

  • 大多数程序都是在windows平台下开发的程序.windows 在现实中也是绕不过的一个系统平台,做为受过几年VC,MFC”虐待”的程序员,在做为一个程序员之前是一位Linux重度使用者,受够了MFC之后一直想要找一个框架替换,使用过GTK,wxWidgets,Qt,最后还是Qt用得多一些.我认为程序跨平台应该是一个基本标准,同一份代码不需改动,或者改动极少,放在不同的平台下编译就能使用.不同平台,同样的界面,同样的操作,同样的体验.这里要讲的是我如何在Linux 下开发跨平台的Qt程序,如何在linux开发windows程序.

MXE (M cross environment)

  • MXE 真的是一个了不起的项目,详情请查看上面链接,非常详细.因为MXE就是在Linux下静态编译的跨平台环境,不需要你一步一步繁杂的编译工具链,直接按教程一键make就下载安装好,但是对Linux的操作熟悉会更加好排错.

安装教程

  • 尽量查看英文教程,这里简单记录一在debian8 x86_64下的安装过程.遇到问题找google.
1
2
3
4
5
6
7

apt-get install \
autoconf automake autopoint bash bison bzip2 flex gettext\
git g++ gperf intltool libffi-dev libgdk-pixbuf2.0-dev \
libtool libltdl-dev libssl-dev libxml-parser-perl make \
openssl p7zip-full patch perl pkg-config python ruby scons \
sed unzip wget xz-utils g++-multilib libc6-dev-i386 libtool-bin

获取最新版本

1
2
3
~$ cd /opt
~$ git clone https://github.com/mxe/mxe.git
~$ make MXE_TARGETS='x86_64-w64-mingw32.static i686-w64-mingw32.static' qt5
  • 如果中间不出错,安装完成如下. i686-w64-mingw32.static目录下就在如:我电脑系统64位下编译32位的exe, 而x86_64-w64-mingw32.static就是编译生成64位的exe.
1
2
$ls  /opt/mxe/usr/
bin i686-w64-mingw32.static include installed lib libexec share x86_64-unknown-linux-gnu x86_64-w64-mingw32.static

设置Qt

  • 英文教程有详细教程如何使用,如:(autotools,CMake,Makefile,QT…),这里贴图记录一下在QtCreator 下的设置.

  • 设置编译器

  • 设置Qt版本

  • 添加Qt Kit

  • 现在就可以在Qt Creator里的工程下添加这个Kit里了,因为这里只是静态编了Release版本,如果指定为Debug配置,编译会出错的.

一些Qt 工程问题

Linux GL 链接库错误.

  • 如果在Linux遇到链接系统下的GL库出错,只要在.pro 加入QMAKE_LIBS_OPENGL -= -lGL,重新qmake,编译.

Release版本加入调试符号

  • 在实际开发有遇到这种问题,在Linux的编译Debug,Release调试与运行都没有错误,但是在windows平台下运就崩溃出错,这时调试最有效的调试手段是是加入调试符号信息,在windows下运行GDB加载该程序,查看出错的原因.pro \
  • 在.pro中在加如下三行,重新qmake,编译.带调试信息的exe会长到几十M.
1
2
3
QMAKE_CXXFLAGS_RELEASE += -g 
QMAKE_CFLAGS_RELEASE += -g
QMAKE_LFLAGS_RELEASE =

谢谢支持

  • 本示例接合vsftpd与Django共用Sqlite3(不限于sqlite3,Mysql,Postgresql)中的数据表认证做一个Web端FTP管理的应用.可以支持多用户,Web端管理帐户,上传文件,Vsftpd端只可以下载当前用户目录下的内容,
  • Web端的上传目录与Vsftpd下载目录映射到服务器的同一个目录.

Vsftpd

  • 这里使用vsftpd-3.0.2版本,这里要对它的源做一小修改,确保每一个用户登录成功之后自动创建一个名为用户名目录.这个目录专属于这个用户,用户上传下载也只限于这个目录下.

  • 源代码位置在vsftpd-3.0.2/twoprocess.c 里的 common_do_login 函数中.如下面代码片段.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
385    static void
386 common_do_login(struct vsf_session* p_sess, const struct mystr* p_user_str,
387 int do_chroot, int anon)
388 {
389 int was_anon = anon;
390 const struct mystr* p_orig_user_str = p_user_str;
391 int newpid;
392 vsf_sysutil_install_null_sighandler(kVSFSysUtilSigCHLD);
393 /* Tells the pre-login child all is OK (it may exit in response) */
394 priv_sock_send_result(p_sess->parent_fd, PRIV_SOCK_RESULT_OK);
395 if (!p_sess->control_use_ssl)
396 {
397 (void) vsf_sysutil_wait();
398 }
399 else
400 {
401 p_sess->ssl_slave_active = 1;
402 }
403 /* Handle loading per-user config options */
404 handle_per_user_config(p_user_str);
405 /* Set this before we fork */
406 p_sess->is_anonymous = anon;
407 priv_sock_close(p_sess);
408 priv_sock_init(p_sess);
409 vsf_sysutil_install_sighandler(kVSFSysUtilSigCHLD, handle_sigchld, 0, 1);
410 if (tunable_isolate_network && !tunable_port_promiscuous)
411 {
412 newpid = vsf_sysutil_fork_newnet();
413 }
414 else
415 {
416 newpid = vsf_sysutil_fork();
417 }
418 if (newpid == 0)
419 {
420 struct mystr guest_user_str = INIT_MYSTR;
421 struct mystr chroot_str = INIT_MYSTR;
422 struct mystr chdir_str = INIT_MYSTR;
423 struct mystr userdir_str = INIT_MYSTR;
424 unsigned int secutil_option = VSF_SECUTIL_OPTION_USE_GROUPS |
425 VSF_SECUTIL_OPTION_NO_PROCS;
426 /* Child - drop privs and start proper FTP! */
427 /* This PR_SET_PDEATHSIG doesn't work for all possible process tree setups.
428 * The other cases are taken care of by a shutdown() of the command
429 * connection in our SIGTERM handler.
430 */
431 vsf_set_die_if_parent_dies();
432 priv_sock_set_child_context(p_sess);
433 if (tunable_guest_enable && !anon)
434 {
435 p_sess->is_guest = 1;
436 /* Remap to the guest user */
437 if (tunable_guest_username)
438 {
439 str_alloc_text(&guest_user_str, tunable_guest_username);
440 }
441 p_user_str = &guest_user_str;
442 if (!tunable_virtual_use_local_privs)
443 {
444 anon = 1;
445 do_chroot = 1;
446 }
447 }
448 if (do_chroot)
449 {
450 secutil_option |= VSF_SECUTIL_OPTION_CHROOT;
451 }
452 if (!anon)
453 {
454 secutil_option |= VSF_SECUTIL_OPTION_CHANGE_EUID;
455 }
456 if (!was_anon && tunable_allow_writeable_chroot)
457 {
458 secutil_option |= VSF_SECUTIL_OPTION_ALLOW_WRITEABLE_ROOT;
459 }
460 calculate_chdir_dir(was_anon, &userdir_str, &chroot_str, &chdir_str,
461 p_user_str, p_orig_user_str);
462
463
/* 通过我的调式这里加下面这一句,用户通过FTP端户登录成功就会创建它自已的目录*/
464 str_mkdir(&chroot_str, 0777);
465
466 chown(str_getbuf(&chroot_str),p_sess->guest_user_uid,p_sess->guest_user_uid);
467
468
469 vsf_secutil_change_credentials(p_user_str, &userdir_str, &chroot_str,
470 0, secutil_option);
471 if (!str_isempty(&chdir_str))
472 {
473 (void) str_chdir(&chdir_str);
474 }
475 str_free(&guest_user_str);
476 str_free(&chroot_str);
477 str_free(&chdir_str);
478 str_free(&userdir_str);
479 p_sess->is_anonymous = anon;
480 seccomp_sandbox_init();
481 seccomp_sandbox_setup_postlogin(p_sess);
482 seccomp_sandbox_lockdown();
483
484 process_post_login(p_sess);
485 bug("should not get here: common_do_login");
486 }
487 /* Parent */
488 priv_sock_set_parent_context(p_sess);
489 if (tunable_ssl_enable)
490 {
491 ssl_comm_channel_set_producer_context(p_sess);
492 }
493 /* The seccomp sandbox lockdown for the priv parent is done inside here */
494 vsf_priv_parent_postlogin(p_sess);
495 bug("should not get here in common_do_login");
496 }

  • 参照vsftpd-3.0.2/INSTALL 直接编译安装.

配置vsftpd

  • /etc/vsftpd.conf 修改如下:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
grep -v '^#' /etc/vsftpd.conf 
cmds_allowed=ABOR,CWD,LIST,NLST,PWD,QUIT,RETR,SIZE,TYPE,USER,ACCT,CDUP,MODE,SYST,FEAT,PASV
listen=YES
listen_ipv6=NO
anonymous_enable=NO
local_enable=YES
write_enable=NO

dirmessage_enable=YES
use_localtime=YES
xferlog_enable=NO
connect_from_port_20=YES
idle_session_timeout=600
data_connection_timeout=120
nopriv_user=vsftpd
chroot_local_user=YES
pam_service_name=vsftpd
user_sub_token=$USER
guest_enable=YES
guest_username=vsftpd
local_root=/opt/data/ftproot/$USER #所有用户目录会在这个目录下.
chroot_local_user=YES
allow_writeable_chroot=YES
chroot_list_enable=NO
virtual_use_local_privs=YES
file_open_mode=0770
anon_mkdir_write_enable=NO
ssl_enable=NO
max_per_ip=65536
log_ftp_protocol=YES
  • /etc/pam.d/vsftpd 修改如下:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Standard behaviour for ftpd(8).
#auth required pam_listfile.so item=user sense=deny file=/etc/ftpusers onerr=succeed

# Note: vsftpd handles anonymous logins on its own. Do not enable pam_ftp.so.

# Standard pam includes
#@include common-account
#@include common-session
#@include common-auth
#auth required pam_shells.so
# 如果使用其它数据库如:postgresql ,把pam_sqlite3.so替换掉
auth required /lib/x86_64-linux-gnu/security/pam_sqlite3.so
account required /lib/x86_64-linux-gnu/security/pam_sqlite3.so
password required /lib/x86_64-linux-gnu/security/pam_sqlite3.so

pam_sqlite-0.3

  • 在github上找到两个项目**libpam-sqlite,dtjm/pam_sqlite3**是关于pam_sqlite3认证的,两个项目都很老了,
  • 这两个代码应该是大同小异,我没有对比,这里用是用libpam-sqlite.也修了源码才可以使用.pam_sqlite3没有测试,应该也是可以的.
1
$ git clone git@github.com:sangeeths/libpam-sqlite.git
  • 这里的修改不一定是通用的可能要根据调试结果来决定,我这里修改是为了迎合后面的Django项目登录认证的.修改的源代如下:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
~/libpam-sqlite# git diff
diff --git a/pam_sqlite3.c b/pam_sqlite3.c
index 565ce30..f2545a4 100644
--- a/pam_sqlite3.c
+++ b/pam_sqlite3.c
@@ -455,7 +455,9 @@ static int auth_verify_password(const char *un,
}

/* get the encrypted password from the database */
+ //const char *stored_pwd = (char*)sqlite3_column_text(stmt, 0);
const char *stored_pwd = (char *)sqlite3_column_text(stmt, 0);
+ stored_pwd = stored_pwd +7; // skip the crypt$$

result = PAM_AUTH_ERR;
switch(options->pw_type) {
@@ -581,7 +583,7 @@ PAM_EXTERN int pam_sm_acct_mgmt(pam_handle_t *pamh,
/* if new password is required then newtok_column = 'y' or '1' */
if(options->newtok_column || options->sql_check_newtok) {
query = format_query(options->sql_check_newtok ? options->sql_check_newtok :
- "SELECT 1 FROM %Ot WHERE %Ou='%U' AND (%On='y' OR %On='1')",
+ "SELECT 1 FROM %Ot WHERE %Ou='%U' AND (%On='y' OR %On='0')",
options, un, NULL);
DBGLOG("query: %s", query);

  • 参照源码目录下的README编译与设置

pam_sqlite.conf

1
2
3
4
5
6
7
$cat /etc/pam_sqlite.conf 
database = /opt/data/db.sqlite3
table = ftp_ftpuser
user_column = email
pwd_column = password
newtok_column = is_staff
pw_type = crypt

Django配置

Sqlite3表结构

1
2
3
sqlite> .schema ftp_ftpuser
CREATE TABLE "ftp_ftpuser" ("password" varchar(128) NOT NULL, "last_login" datetime NULL, "is_superuser" bool NOT NULL, "name" varchar(256) NOT NULL, "email" varchar(254) NOT NULL PRIMARY KEY, "phone_num" varchar(15) NOT NULL, "reg_ip" char(39) NOT NULL, "is_active" bool NOT NULL, "is_staff" bool NOT NULL, "date_joined" datetime NOT NULL, "last_ip" char(39) NOT NULL);
CREATE UNIQUE INDEX "ftp_ftpuser_name_2294ce6a_uniq" ON "ftp_ftpuser" ("name", "email");

Settings

  • 在Django项目里的settings.py 添加如下代码片段.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# Application definition
PASSWORD_HASHERS = (
'django.contrib.auth.hashers.CryptPasswordHasher',
)

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'), #这里的**db.sqlite3** 应该与**/etc/pam_sqlite.conf**里对应
}
}


# Password validation
# https://docs.djangoproject.com/en/1.9/ref/settings/#authassword-validators

AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]

#AUTHENTICATION_BACKENDS = ['ftp.auth.CustomBackEnd']
AUTH_USER_MODEL='ftp.FtpUser'
MEDIA_ROOT='/opt/data/ftproot' #这里的路径与vsftpd.conf是对应的.这里Django 程序创建的目录就会在这个目录下面

  • Django 程序创建目录的代码片段如下:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
def get_upload_dir(req,filename):
tname = req.target_name
pname = tname.product_name
fdir = '/'.join([settings.MEDIA_ROOT,pname.ftp_user.email,pname.name,tname.name,req.ver_name])
#fdir = '/'.join([settings.MEDIA_ROOT,product.ftp_user.email,req.name.name,req.target_name.target_name,req.ver_name])
fdir = unicode(fdir)
if os.path.exists(fdir):
try:
os.makedirs(fdir)
except OSError:
pass
filepath = '/'.join([fdir,req.file_name.name])
if os.path.isfile(filepath):
os.remove(filepath)
return filepath

总结

  • 这里的数据库表结构都是通过Django的模型创建的,通过Django端注册用户,设置相应权限(如是否允许该用户登录等.),FTP端就可以使用该用户登录服务器进行下载.具体应用要根据需求灵活改变,这里只是做了一个简单示例,
  • 也没有添加SSL加密连接.后续加入SSL支持.

谢谢支持

GPG error

  • W: GPG error: http://mirrors.ustc.edu.cn bullseye-updates InRelease: The following signatures couldn’t be verified because the public key is not available: NO_PUBKEY 648ACFD622F3D138 NO_PUBKEY 0E98404D386FA1D9
1
~$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 0E98404D386FA1D9 648ACFD622F3D138

DEPRECATION legacy keyring

1
2
W: http://dl.google.com/linux/earth/deb/dists/stable/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.

  • list all keys
1
2
3
4
5
6
7
8
9
10
11
12
13
14
~$ apt-key --keyring /etc/apt/trusted.gpg list | grep "google" -A +5 -B +5
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
uid [ unknown] NodeSource <gpg@nodesource.com>
sub rsa4096 2014-06-13 [E]

pub rsa4096 2016-04-12 [SC]
EB4C 1BFD 4F04 2F6D DDCC EC91 7721 F63B D38B 4796
uid [ unknown] Google Inc. (Linux Packages Signing Authority) <linux-packages-keymaster@google.com>
sub rsa4096 2021-10-26 [S] [expires: 2024-10-25]

pub rsa4096 2015-04-17 [SC] [expired: 2020-04-15]
1A4C 919D B987 D435 9396 38B9 1421 9A96 E15E 78F4
uid [ expired] GitLab B.V. (package repository signing key) <packages@gitlab.com>

  • export to a new file.
    1
    2
    3
    ~$ sudo apt-key export D38B4796 | sudo gpg --dearmour -o /etc/apt/trusted.gpg.d/dl-google.gpg
    Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).

PulseAudio 设置与应用

  • 安装如下包,具体安装可参考这里

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    ~$ dpkg -l | grep "pulseaudio"
    ii gstreamer1.0-pulseaudio:amd64 1.10.4-1 amd64 GStreamer plugin for PulseAudio
    ii pulseaudio 10.0-1+deb9u1 amd64 PulseAudio sound server
    ii pulseaudio-dlna 0.5.2-1 all Stream audio to DLNA devices and Chromecasts
    ii pulseaudio-module-bluetooth 10.0-1+deb9u1 amd64 Bluetooth module for PulseAudio sound server
    ii pulseaudio-module-gconf 10.0-1+deb9u1 amd64 GConf module for PulseAudio sound server
    ii pulseaudio-module-jack 10.0-1+deb9u1 amd64 jackd modules for PulseAudio sound server
    ii pulseaudio-utils 10.0-1+deb9u1 amd64 Command line tools for the PulseAudio sound server
    ii alsa-tools 1.1.3-1 amd64 Console based ALSA utilities for specific hardware
    ii alsa-utils 1.1.3-1 amd64 Utilities for configuring and using ALSA
    ii alsamixergui 0.9.0rc2-1-10 amd64 graphical soundcard mixer for ALSA soundcard driver
  • 添加这一行load-module module-alsa-sink device=hw:1,0/etc/pulse/default.pa里面.

1
2
3
4
~$ cat /etc/pulse/daemon.conf
daemonize = yes
allow-module-loading = yes
system-instance = yes
  • 把下面的输出转到 PulseAudio.
1
2
3
4
5
6
7
8
9
10
11
12
13
~$ cat ~/.asoundrc
pcm.pulse {
type pulse
}
ctl.pulse {
type pulse
}
pcm.!default {
type pulse
}
ctl.!default {
type pulse
}
  • 启动,停止 PulseAudio
1
2
3
4
5
6
7
8
9
10
11
12
13
14
~$ pulseaudio --kill
~$ pulseaudio --start

~$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: HDMI [HDA ATI HDMI], device 3: HDMI 0 [HDMI 0]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: Generic [HD-Audio Generic], device 0: ALC887-VD Analog [ALC887-VD Analog]
Subdevices: 0/1
Subdevice #0: subdevice #0
card 1: Generic [HD-Audio Generic], device 1: ALC887-VD Digital [ALC887-VD Digital]
Subdevices: 1/1
Subdevice #0: subdevice #0
  • 如果出现任何问题,仔细查看系统日志解决.

minicom 使用

  • Linux下的超级终端工具minicom 是命令行的工具不像 windows 下的串口工具那么直观,但是使用习惯了就一样.
  • 初次使用minicom除了一些串口的常规设置,跟其它串口工具都是一样的.
  • 在当前 Shell下输入minicom -s会出现如下界面:
1
2
3
4
5
6
7
8
9
10
11
+-----[configuration]------+
| Filenames and paths |
| File transfer protocols |
| Serial port setup |
| Modem and dialing |
| Screen and keyboard |
| Save setup as dfl |
| Save setup as.. |
| Exit |
| Exit from Minicom |
+--------------------------+
  • 选择Serial port setup 设置串口参数:
1
2
3
4
5
6
7
8
9
10
11
+-----------------------------------------------------------------------+
| A - Serial Device : /dev/ttyUSB5 |
| B - Lockfile Location : /var/lock |
| C - Callin Program : |
| D - Callout Program : |
| E - Bps/Par/Bits : 115200 8N1 |
| F - Hardware Flow Control : No |
| G - Software Flow Control : No |
| |
| Change which setting? |
+-----------------------------------------------------------------------+
  • 设置完成回车返回到一个菜单.选择Save setup as dfl 保存退出.

  • 我一般使用 minicom 的参数去连接.例如:

1
~$  minicom -o -b 115200 -D /dev/ttyUSB5

AT 命令

esptool 升级固件

  • 参考链接:

  • ESP8266 Tools

  • ESP8266 Wiki

  • ESP8266 SDK

  • 使用 Arduino 更新 ESP8266 firmware

  • ESP-01为例.

    1
    2
    3
    4
    5
    6
    7
    8
    GPIO0 --> GND
    GPIO2 --> 3V3
    CH_PD --> 3V3
    RST --> 3V3
    VCC --> 3V3
    GND --> GND
    TX --> UART(RX)
    RX --> UART(TX)
  • esptool 工具

    1
    ~$ esptool.py   -p /dev/ttyUSB5 write_flash --flash_mode dio --flash_size 8m 0x0 AiThinker_ESP8266_DIO_8M_8M_20160615_V1.5.4.bin
  • 这个升级问题折腾了我很久,今天总算是升级成功.它们的文档很杂,图与板子上的名称又不全部对应.

下面是更新部分

  • Updating ESP8266 Firmware
  • Espressif Systems
  • Update the Firmware in Your ESP8266 Wi-Fi Module
  • ESP8266 - Upgrading the Firmware
  • 这里以下载最新的v1.7.4固件为例,注意查看各个子文件目录里的README.md.
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66

    TTL-USB ESP01

    RX ---> TX
    TX ---> RX
    GND ---> GND
    GND ---> GPIO0 # 烧写完成后,把它移除就进入正常模式
    3V3 ---> 3V3
    3V3 ---> CH_PD

    ~$ esptool.py flash_id
    esptool.py v3.0-dev
    Found 2 serial ports
    Serial port /dev/ttyUSB1
    Connecting....
    Detecting chip type... ESP8266
    Chip is ESP8266EX
    Features: WiFi
    Crystal is 26MHz
    MAC: 5c:cf:7f:0a:6c:b8
    Uploading stub...
    Running stub...
    Stub running...
    Manufacturer: e0
    Device: 4014
    Detected flash size: 1MB
    Hard resetting via RTS pin...

    ~$ esptool.py -p /dev/ttyUSB0 write_flash --flash_mode qio --flash_freq 40m 0x00000 boot_v1.7.bin 0x01000 at/512+512/user1.1024.new.2.bin 0xfc000 esp_init_data_default_v08.bin 0x7e000 blank.bin 0xfe000 blank.bin 0xfb000 blank.bin

    esptool.py v3.0-dev
    Serial port /dev/ttyUSB0
    Connecting....
    Detecting chip type... ESP8266
    Chip is ESP8266EX
    Features: WiFi
    Crystal is 26MHz
    MAC: 5c:cf:7f:0a:6c:b8
    Uploading stub...
    Running stub...
    Stub running...
    Configuring flash size...
    Auto-detected Flash size: 1MB
    Flash params set to 0x0020
    Compressed 4080 bytes to 2936...
    Wrote 4080 bytes (2936 compressed) at 0x00000000 in 0.3 seconds (effective 123.6 kbit/s)...
    Hash of data verified.
    Compressed 413444 bytes to 296966...
    Wrote 413444 bytes (296966 compressed) at 0x00001000 in 26.3 seconds (effective 126.0 kbit/s)...
    Hash of data verified.
    Compressed 128 bytes to 75...
    Wrote 128 bytes (75 compressed) at 0x000fc000 in 0.0 seconds (effective 85.1 kbit/s)...
    Hash of data verified.
    Compressed 4096 bytes to 26...
    Wrote 4096 bytes (26 compressed) at 0x0007e000 in 0.0 seconds (effective 4089.5 kbit/s)...
    Hash of data verified.
    Compressed 4096 bytes to 26...
    Wrote 4096 bytes (26 compressed) at 0x000fe000 in 0.0 seconds (effective 4159.4 kbit/s)...
    Hash of data verified.
    Compressed 4096 bytes to 26...
    Wrote 4096 bytes (26 compressed) at 0x000fb000 in 0.0 seconds (effective 4216.8 kbit/s)...
    Hash of data verified.

    Leaving...
    Hard resetting via RTS pin...

  • 使用minicom时注意,软硬件流控要为NO,具体设置Ctrl-A Z, -> O [Configure Minicom] -> Serial Port Setup -> F.使用putty边接,输入下面的命令
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# 查看固件
at+GMR
AT version:1.7.4.0(May 11 2020 19:13:04)
SDK version:3.0.4(9532ceb)
compile time:May 27 2020 10:12:17
Bin version(Wroom 02):1.7.4
OK

# 扫AP
AT+CWLAP
+CWLAP:(4,"18575654312",-84,"cc:2d:21:64:23:40",1,63,0,4,4,7,0)
+CWLAP:(3,"lcy",-43,"20:0d:b0:40:21:8f",1,75,0,4,4,3,0)
OK

# 看地址
at+CIFSR
+CIFSR:STAIP,"192.168.2.122"
+CIFSR:STAMAC,"5c:cf:7f:0a:6c:b8"

OK

# 看内存
AT+SYSRAM?
+SYSRAM:51952

OK

转换GBK文本文件的问题

  • 有时在Linux下打开GBK编码的文件会出现乱码,优其是一些在window下载打包的源码压缩包,源码文件全都是GBK编码.在Linux下用一些文本编辑器打开全是乱码,有些甚至连目录都是乱码.
  • 下面记录一个shell脚本批量转换成UTF8格式的文件.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ cat convert.sh
#!/bin/bash
TMP_IFS=$IFS
IFS=$(echo -en "\n\b")
for f in `find . -type f`;do
org=$(file $f)
string=$(echo $org | grep "ISO-8859") ;
if [ -n "$string" ]; then
infile=$(echo $string | awk -F':' '{print $1}');
#echo $infile
iconv -f GBK -t UTF8 $infile > 1;
mv 1 $infile;
fi ;
done
IFS=$TMP_IFS
1
2
3
4
5
6
7
#!/bin/bash
TMP_IFS=$IFS
IFS=$(echo -en "\n\b")

for f in `find . \( -iname "*.txt" -o -iname "*.c" -o -iname "*.h" \)`;do
iconv -f GBK -t UTF8 -c $f > 1; mv 1 $f;
done
  • 在转换文件时,有时候路径中会包含空格,如果不按照上面的方式设置IFS变量,脚本就会出错在有空格的地方分隔,认为是两个文件.上面这个脚本只是
  • 一个很简单的处理当前目录下所有文件的转换.还可以把它写的更通用灵活.

批量清除文件名中的特定字符串

  • 有时从网站下载一些文件,资源经会把自已的名字或者网址加在文件名如: 电影名_[www.xxx.com].mkv,这样的格式
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    #!/bin/bash
    TMP_IFS=$IFS
    IFS=$(echo -en "\n\b")
    sub1="www\.xxx\.com"
    sub2="www\.bbb\.com"
    sub3="www\.ccc\.com"
    #sub4="[(]www.ddd.com[)]" # 例如: fname-(www.ddd.com).pdf, sed -re "s/$sub4//g"


    # 例如:匹配 538081 SCALA学习手册.pdf 556997 Scala函数式编.pdf 这样的文件名,删除前面的6个数字
    #
    # reg='^([0-9]{6})'
    # for fname in `ls`;do
    # string=$(echo $fname)
    # if [[ $string =~ $reg ]];
    # then
    # nstr=$(echo $string | sed -r "s/${reg}//g")
    # echo $nstr
    # mv $fname $nstr
    # fi
    # done
    #
    #

    for fname in `ls`;
    do
    string=$(echo $fname)
    if [[ $string = *${sub1}* || $string = *${sub2}* || $string = *${sub3}* || $string = *${sub4}* ]];
    then
    nstring=$(echo $s | sed -e "s/${sub1}//g" -e "s/$sub2//g" -e "s/$sub3//g" -e "s/\(()\)//g/" -e "s/\[[]\]//g" )

    echo $nstring
    mv $f $nstring
    fi
    done

    IFS=$TMP_IFS

路由表内存在多个default dev的问题

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
~$ ip route
0.0.0.0 dev veth49bb6a8 scope link
0.0.0.0 dev eth0 scope link
default dev veth49bb6a8 scope link
default dev eth0 scope link
default via 192.168.1.1 dev br-lan onlink
169.254.0.0/16 dev eth0 proto kernel scope link src 169.254.46.34
169.254.0.0/16 dev veth49bb6a8 proto kernel scope link src 169.254.136.170
169.254.0.0/16 dev br-lan scope link metric 1000
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
192.168.1.0/24 dev br-lan proto kernel scope link src 192.168.1.178

~$ dpkg -l | grep "avahi"
ii avahi-autoipd 0.8-10 amd64 Avahi IPv4LL network address configuration daemon
ii avahi-daemon 0.8-10 amd64 Avahi mDNS/DNS-SD daemon
ii avahi-utils 0.8-10 amd64 Avahi browsing, publishing and discovery utilities
ii libavahi-client3:amd64 0.8-10 amd64 Avahi client library
ii libavahi-common-data:amd64 0.8-10 amd64 Avahi common data files
ii libavahi-common3:amd64 0.8-10 amd64 Avahi common library
ii libavahi-core7:amd64 0.8-10 amd64 Avahi's embeddable mDNS/DNS-SD library
ii libavahi-glib1:amd64 0.8-10 amd64 Avahi GLib integration library

~$ dpkg -l | grep "connman"
ii connman 1.41-3 amd64 Intel Connection Manager daemon
ii connman-gtk 1.1.1+git20180626.b72c6ab-3 amd64 fully-featured GUI for ConnMan with systray support

  • 因为是安装了debian buster lxde-desktop版本,先是怀疑avahi问题,看了它的配置,有去配置169.254.x.x的地址,删除后,还是没有解决问题。列出本地端口时,发现有一个进程listen 127.0.0.1:53, 最终查到是connman导至的问题。

Linux 下wireless WPA加密连接

  • 确保安装了 wpasupplicant

    1
    ~$ sudo apt-get install wpasupplicant.
  • 创建/etc/wpa_supplicant.conf或者从/usr/share/doc/wpa_supplicant/examples/wpa_supplicant.conf.gz复制

  • 例如:

    1
    2
    3
    4
    network={
    ssid="ssid_name"
    psk="password"
    }
  • 手动命令行连接

1
2
~$ sudo wpa_supplicant -B -iwlan0 -c/etc/wpa_supplicant.conf -Dwext
~$ sudo dhclient wlan0

MT7601U 开启软件AP功能(非hostapd)

  • 如果安装或是编译了其它的mt7601u的驱动,需要把它卸载或者加入黑名单里,不能加载它.
    1
    2
    ~$ cat /etc/modprobe.d/blacklist-mt7601u
    blacklist mt7601u

RTL8812AU驱动(0bda:0811)

1
2
3
4
~$ lsusb -s 008 -v

Bus 008 Device 008: ID 0bda:0811 Realtek Semiconductor Corp.

  • 这块RTL8812AU的网卡是EDUP AC-1608,查看它官方驱动是法支持AP功能且还不支持nl80211的驱动,之后去到github搜索到一些驱动,有一些是只能使用网卡功能,有一些还连编译都无法通过.后来验证两个驱动如上,最后使用了aircrack-ng解决了所有的问题,不愧是黒技术的先锋.
  • 按照aircrack-ng的提示,直接make && make install或者sudo ./dkms-install.sh就直接编译并安装到内核树的驱动里.加载成功后,查看模块信息,检测到功能如下:
1
2
~$ modinfo 88XXau | grep "0811"
alias: usb:v0BDAp0811d*dc*dsc*dp*ic*isc*ip*in*
  • 查看phy的信息.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
~$ iw list
Wiphy phy3
max # scan SSIDs: 9
max scan IEs length: 2304 bytes
max # sched scan SSIDs: 0
max # match sets: 0
max # scan plans: 1
max scan plan interval: -1
max scan plan iterations: 0
Retry short limit: 7
Retry long limit: 4
Coverage class: 0 (up to 0m)
Supported Ciphers:
* WEP40 (00-0f-ac:1)
* WEP104 (00-0f-ac:5)
* TKIP (00-0f-ac:2)
* CCMP-128 (00-0f-ac:4)
* CMAC (00-0f-ac:6)
Available Antennas: TX 0x1 RX 0x1
Supported interface modes:
* IBSS
* managed
* AP
* monitor
* P2P-client
* P2P-GO
Band 1:
Capabilities: 0x1972
HT20/HT40
Static SM Power Save
RX Greenfield
RX HT20 SGI
RX HT40 SGI
RX STBC 1-stream
Max AMSDU length: 7935 bytes
DSSS/CCK HT40
Maximum RX AMPDU length 65535 bytes (exponent: 0x003)
Minimum RX AMPDU time spacing: 16 usec (0x07)
HT Max RX data rate: 150 Mbps
HT TX/RX MCS rate indexes supported: 0-7
Bitrates (non-HT):
* 1.0 Mbps
* 2.0 Mbps
* 5.5 Mbps
* 11.0 Mbps
* 6.0 Mbps
* 9.0 Mbps
* 12.0 Mbps
* 18.0 Mbps
* 24.0 Mbps
* 36.0 Mbps
* 48.0 Mbps
* 54.0 Mbps
Frequencies:
* 2412 MHz [1] (20.0 dBm)
* 2417 MHz [2] (20.0 dBm)
* 2422 MHz [3] (20.0 dBm)
* 2427 MHz [4] (20.0 dBm)
* 2432 MHz [5] (20.0 dBm)
* 2437 MHz [6] (20.0 dBm)
* 2442 MHz [7] (20.0 dBm)
* 2447 MHz [8] (20.0 dBm)
* 2452 MHz [9] (20.0 dBm)
* 2457 MHz [10] (20.0 dBm)
* 2462 MHz [11] (20.0 dBm)
* 2467 MHz [12] (20.0 dBm)
* 2472 MHz [13] (20.0 dBm)
* 2484 MHz [14] (20.0 dBm)
Band 2:
Capabilities: 0x1972
HT20/HT40
Static SM Power Save
RX Greenfield
RX HT20 SGI
RX HT40 SGI
RX STBC 1-stream
Max AMSDU length: 7935 bytes
DSSS/CCK HT40
Maximum RX AMPDU length 65535 bytes (exponent: 0x003)
Minimum RX AMPDU time spacing: 16 usec (0x07)
HT Max RX data rate: 150 Mbps
HT TX/RX MCS rate indexes supported: 0-7
VHT Capabilities (0x03c03122):
Max MPDU length: 11454
Supported Channel Width: neither 160 nor 80+80
short GI (80 MHz)
SU Beamformee
+HTC-VHT
VHT RX MCS set:
1 streams: MCS 0-9
2 streams: not supported
3 streams: not supported
4 streams: not supported
5 streams: not supported
6 streams: not supported
7 streams: not supported
8 streams: not supported
VHT RX highest supported: 434 Mbps
VHT TX MCS set:
1 streams: MCS 0-9
2 streams: not supported
3 streams: not supported
4 streams: not supported
5 streams: not supported
6 streams: not supported
7 streams: not supported
8 streams: not supported
VHT TX highest supported: 434 Mbps
Bitrates (non-HT):
* 6.0 Mbps
* 9.0 Mbps
* 12.0 Mbps
* 18.0 Mbps
* 24.0 Mbps
* 36.0 Mbps
* 48.0 Mbps
* 54.0 Mbps
Frequencies:
* 5180 MHz [36] (23.0 dBm)
* 5200 MHz [40] (23.0 dBm)
* 5220 MHz [44] (23.0 dBm)
* 5240 MHz [48] (23.0 dBm)
* 5260 MHz [52] (23.0 dBm) (radar detection)
* 5280 MHz [56] (23.0 dBm) (radar detection)
* 5300 MHz [60] (23.0 dBm) (radar detection)
* 5320 MHz [64] (23.0 dBm) (radar detection)
* 5500 MHz [100] (23.0 dBm) (radar detection)
* 5520 MHz [104] (23.0 dBm) (radar detection)
* 5540 MHz [108] (23.0 dBm) (radar detection)
* 5560 MHz [112] (23.0 dBm) (radar detection)
* 5580 MHz [116] (23.0 dBm) (radar detection)
* 5600 MHz [120] (23.0 dBm) (radar detection)
* 5620 MHz [124] (23.0 dBm) (radar detection)
* 5640 MHz [128] (23.0 dBm) (radar detection)
* 5660 MHz [132] (23.0 dBm) (radar detection)
* 5680 MHz [136] (23.0 dBm) (radar detection)
* 5700 MHz [140] (23.0 dBm) (radar detection)
* 5720 MHz [144] (23.0 dBm) (radar detection)
* 5745 MHz [149] (30.0 dBm)
* 5765 MHz [153] (30.0 dBm)
* 5785 MHz [157] (30.0 dBm)
* 5805 MHz [161] (30.0 dBm)
* 5825 MHz [165] (30.0 dBm)
* 5845 MHz [169] (30.0 dBm)
* 5865 MHz [173] (30.0 dBm)
* 5885 MHz [177] (30.0 dBm)
Supported commands:
* new_interface
* set_interface
* new_key
* start_ap
* new_station
* set_bss
* join_ibss
* set_pmksa
* del_pmksa
* flush_pmksa
* remain_on_channel
* frame
* set_channel
* connect
* disconnect
Supported TX frame types:
* IBSS: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70 0x80 0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
* managed: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70 0x80 0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
* AP: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70 0x80 0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
* AP/VLAN: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70 0x80 0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
* P2P-client: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70 0x80 0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
* P2P-GO: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70 0x80 0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
Supported RX frame types:
* IBSS: 0xd0
* managed: 0x40 0xb0 0xd0
* AP: 0x00 0x20 0x40 0xa0 0xb0 0xc0 0xd0
* AP/VLAN: 0x00 0x20 0x40 0xa0 0xb0 0xc0 0xd0
* P2P-client: 0x40 0xd0
* P2P-GO: 0x00 0x20 0x40 0xa0 0xb0 0xc0 0xd0
WoWLAN support:
* wake up on anything (device continues operating normally)
software interface modes (can always be added):
* monitor
interface combinations are not supported
Device supports SAE with AUTHENTICATE command
Device supports scan flush.
Supported extended features:
michael@debian:~/3TB-DISK$
michael@debian:~/3TB-DISK$ iw wlan0 info
Interface wlan0
ifindex 13
wdev 0x300000001
addr e8:4e:06:5b:fa:98
type managed
wiphy 3
txpower 20.00 dBm

使用wpa_supplicant

  • wpa_supplicant is a WPA Supplicant for Linux, BSD, Mac OS X, and Windows with support for WPA and WPA2 (IEEE 802.11i / RSN).
  • wpa_supplicant是一个提供WPA认证的客户端,它除了可以做为网卡端提供认证外,还可以在AP模式提供认证.

AP模式

1
2
3
4
5
6
7
8
9
10
~$ cat /etc/wpa_supplicant/wpa_supplicant.conf
network={
ssid="RPi3B"
mode=2 # AP 模式.需要网卡支持.
proto=RSN
key_mgmt=WPA-PSK
pairwise=CCMP TKIP
group=CCMP TKIP
psk="12345678"
}
  • 创建如上面的配置文件,就可以使用命令运行它成为一AP点.认证连接后,需要为客户端分配IP,具体可以在本机上安装一个udhcpd.也可以客户端手动设置IP与这边的AP端是在同一个网络里.
1
2
3
4
5
6
7
8
9
10
~$ sudo wpa_supplicant -Dnl80211 -iwlan0 -c/etc/wpa_supplicant/wpa_supplicant.conf
# 如果加 -B 就是运行在后台模式.
Successfully initialized wpa_supplicant
# 这个错误是内核里没有选择`RF switch subsystem support`编译
rfkill: Cannot open RFKILL control device
Using interface wlan0 with hwaddr e8:xx:06:xx:xx:98 and ssid "RPi3B"
wlan0: interface state UNINITIALIZED->ENABLED
wlan0: AP-ENABLED
wlan0: CTRL-EVENT-CONNECTED - Connection to e8:xx:06:xx:xx:98 completed [id=0 id_str=]

  • 也可把它集成到/etc/network/interfaces里,网卡开启之后自动成为一个AP点.
1
2
3
4
5
6
7
8
9
10
~$ cat /etc/network/interfaces
auto wlan0
iface wlan0 inet static
address 192.168.1.100
netmask 255.255.255.0
network 192.168.1.0
gateway 192.168.1.1
pre-up wpa_supplicant -B -Dnl80211 -iwlan0 -c/etc/wpa_supplicant/wpa_supplicant.conf
post-down killall -q wpa_supplicant
wait-delay 15

RTL8169 驱动

  • No Ethernet Connection in Ubuntu 16.04/Linux Mint 18 with Realtek RTL8111/8168/8411

  • r8168 module not used by ethernet controller

  • https://github.com/fintecheando/r8168

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    #  查看本机硬件信息.
    ~$ lshw -c network
    *-network UNCLAIMED
    description: Ethernet interface
    product: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
    vendor: Realtek Semiconductor Co., Ltd.
    physical id: 0
    bus info: pci@0000:02:00.0
    [....]


    ~$ tree r8168-8.043.02/
    r8168-8.043.02/
    ├── dkms.conf
    ├── Makefile
    ├── r8168_asf.c
    ├── r8168_asf.h
    ├── r8168_dash.h
    ├── r8168_fiber.h
    ├── r8168.h
    ├── r8168_n.c
    ├── r8168_realwow.h
    ├── rtl_eeprom.c
    ├── rtl_eeprom.h
    ├── rtltool.c
    └── rtltool.h

    0 directories, 13 files

    # 查看系统已经添加了那些dkms包.
    ~$ cd /usr/src/
    lcy@debian:/usr/src$ dkms status
    aufs, 4.9+20161219, 4.9.0-7-amd64, x86_64: installed
    aufs, 4.9+20161219, 4.9.0-8-amd64, x86_64: installed
    nvidia, 410.48, 4.14.77-20181016-lcy-amd64, x86_64: installed
    r8168, 8.043.02, 4.9.0-8-amd64, x86_64: installed
    rtl8812au, 4.3.8.12175.20140902+dfsg, 4.14.77-20181016-lcy-amd64, x86_64: installed
    rtl8812au, 4.3.8.12175.20140902+dfsg, 4.9.0-8-amd64, x86_64: installed

    # 如果没有看到添加进来,先把源码目录复制到/usr/src/r8168-8.043.02,再用下面命令加入dkms.
    ~$ sudo dkms add -m r8168 -v 8.043.02

    # 使用dkms编译驱动包.
    ~$ sudo dkms build -m r8168 -v 8.043.02

    Kernel preparation unnecessary for this kernel. Skipping...

    Building module:
    cleaning build area...
    make -j6 KERNELRELEASE=4.14.77-20181016-lcy-amd64 -C /lib/modules/4.14.77-20181016-lcy-amd64/build M=/var/lib/dkms/r8168/8.043.02/build...(bad exit status: 2)
    Error! Bad return status for module build on kernel: 4.14.77-20181016-lcy-amd64 (x86_64)
    Consult /var/lib/dkms/r8168/8.043.02/build/make.log for more information.

    # 查看dkms编译为什么出错.
    ~$ cat /var/lib/dkms/r8168/8.043.02/build/make.log
    DKMS make.log for r8168-8.043.02 for kernel 4.14.77-20181016-lcy-amd64 (x86_64)
    Sat Dec 29 16:31:32 CST 2018
    make: Entering directory '/usr/src/linux-4.14.77'
    AR /var/lib/dkms/r8168/8.043.02/build/built-in.o
    CC [M] /var/lib/dkms/r8168/8.043.02/build/r8168_n.o
    CC [M] /var/lib/dkms/r8168/8.043.02/build/r8168_asf.o
    CC [M] /var/lib/dkms/r8168/8.043.02/build/rtl_eeprom.o
    CC [M] /var/lib/dkms/r8168/8.043.02/build/rtltool.o
    /var/lib/dkms/r8168/8.043.02/build/r8168_n.c: In function ‘rtl8168_rx_interrupt’:
    /var/lib/dkms/r8168/8.043.02/build/r8168_n.c:25418:28: error: ‘struct net_device’ has no member named ‘last_rx’
    dev->last_rx = jiffies;
    ^~
    scripts/Makefile.build:328: recipe for target '/var/lib/dkms/r8168/8.043.02/build/r8168_n.o' failed
    make[1]: *** [/var/lib/dkms/r8168/8.043.02/build/r8168_n.o] Error 1
    Makefile:1527: recipe for target '_module_/var/lib/dkms/r8168/8.043.02/build' failed
    make: *** [_module_/var/lib/dkms/r8168/8.043.02/build] Error 2
    make: Leaving directory '/usr/src/linux-4.14.77'
  • 删除原有的系统包

1
2
~$ sudo apt-get purge r8168-dkms
~$ sudo sh -c ‘echo blacklist r8169 >> /etc/modprobe.d/blacklist.conf’
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

~$ git clone https://github.com/fintecheando/r8168
~$ cd r8168
~$ sudo ./autorun.sh
# 查看内核参数
~$ lsmod | grep r8168
r8168 520192 0

# 查看网卡参数
~$ ethtool -i eth0
driver: r8168
version: 8.045.08-NAPI
firmware-version:
expansion-rom-version:
bus-info: 0000:02:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no
1
2
3
4
5
~$ dmesg
[ 4.927167] iwlwifi 0000:02:00.0 wlp2s0: renamed from wlan0

~$ sudo ln -nfs /dev/null /etc/systemd/network/99-default.link
~$ sudo udevadm trigger

使用hostapd配置WIFI热点

查看网卡配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
~$ iw list | grep "Supported interface modes" -A 8
Supported interface modes:
* IBSS
* managed
* AP
* monitor
* P2P-client
* P2P-GO
Band 1:
Capabilities: 0x1862
~$ iw reg get
global
country 00: DFS-UNSET
(2402 - 2472 @ 40), (N/A, 20), (N/A)
(2457 - 2482 @ 20), (N/A, 20), (N/A), AUTO-BW, NO-IR
(2474 - 2494 @ 20), (N/A, 20), (N/A), NO-OFDM, NO-IR
(5170 - 5250 @ 80), (N/A, 20), (N/A), AUTO-BW, NO-IR
(5250 - 5330 @ 80), (N/A, 20), (0 ms), DFS, AUTO-BW, NO-IR
(5490 - 5730 @ 160), (N/A, 20), (0 ms), DFS, NO-IR
(5735 - 5835 @ 80), (N/A, 20), (N/A), NO-IR
(57240 - 63720 @ 2160), (N/A, 0), (N/A)

安装

1
2
3
~$ sudo apt-get install hostapd
~$ cp /usr/share/doc/hostapd/examples/hostapd.config.gz /etc/hostapd/
~$ gzip -d /etc/hostapd/hostapd.conf

调试配置文件错误

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# 配置文件错误
~$ hostapd /etc/hostapd/hostapd.conf
Configuration file: /etc/hostapd/hostapd.conf
Cannot enable IEEE 802.11d without setting the country_code
2 errors found in configuration file '/etc/hostapd/hostapd.conf'
Failed to set up interface with /etc/hostapd/hostapd.conf
Failed to initialize interface

# 缺少rfkill模块支持.
~$ sudo hostapd -dd /etc/hostapd/hostapd.conf
random: Trying to read entropy from /dev/random
Configuration file: /etc/hostapd/hostapd.conf
ctrl_interface_group=0
rfkill: Cannot open RFKILL control device
nl80211: RFKILL status not available

# 测试wpa_support
~$ sudo cp /usr/share/doc/wpasupplicant/examples/wpa_supplicant.conf.gz /etc/wpa_supplicant/
~$ wpa_supplicant -B -i wlan0 -D wext -c /etc/wpa_supplicant/wpa_supplicant.conf

配置实例

  • Hostapd
  • 下面的配置有一些网卡是不支持的,出错需注释相关行.
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    ~$ sudo grep -v '^#' hostapd.conf | grep -v '^$'
    interface=wlan0
    driver=nl80211
    logger_syslog=1
    logger_syslog_level=0
    logger_stdout=1
    logger_stdout_level=2
    ctrl_interface=/var/run/hostapd
    ctrl_interface_group=0
    ssid=lcy_home
    country_code=CN
    ieee80211d=1
    hw_mode=g
    channel=4
    auth_algs=3
    ignore_broadcast_ssid=0
    uapsd_advertisement_enabled=1
    ieee80211n=1
    ieee80211ac=1
    eap_server=0
    own_ip_addr=127.0.0.1
    wpa=2
    wpa_psk_file=/etc/hostapd.wpa_psk
    wpa_key_mgmt=WPA-PSK
    wpa_pairwise=TKIP
    rsn_pairwise=CCMP
    wpa_group_rekey=600
    wpa_gmk_rekey=86400

配置主机路由

1
2
root# echo "net.ipv4.ip_forward=1" > /etc/sysconf.ctl  # 这一行很重要.
root# iptables -t nat -I POSTROUTING -j MASQUERADE # 配置主机的NAT路由.确定是在第一条规则.

Linux 下是使用蓝牙串口通信

  • 如果是自编译内核要确保RFCOMM protocol support (BT_RFCOMM)是编译成模块的,而不能直接编入内核或者不编.
  • Linux下使用蓝牙工具要安装bluez软件,有一个图形配置程序blueman,使用会报一些PythonDbus的错误,还是直接修改配置文件解决使用问题.
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    ~$ cat /etc/bluetooth/rfcomm.conf
    rfcomm0 {
    bind yes;
    device 00:15:83:00:43:AB
    channel 0;
    comment "Serial Port";
    }


    rfcomm1 {
    bind yes;
    device 00:14:01:22:14:49;
    channel 1;
    connment "Serial Port 1";
    }

配对蓝牙

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ bluetoothctl
[NEW] Controller 00:02:5B:0A:59:3A debian-0 [default]
[NEW] Device 00:14:01:22:14:49 lcy
[bluetooth]# power on
Changing power on succeeded
[bluetooth]# agent on
Agent registered
[bluetooth]# pair 00:14:01:22:14:49
Attempting to pair with 00:14:01:22:14:49
[CHG] Device 00:14:01:22:14:49 Connected: yes
Request PIN code
[agent] Enter PIN code: 1234
[CHG] Device 00:14:01:22:14:49 Paired: yes
Pairing successful
[CHG] Device 00:14:01:22:14:49 Connected: no

手动连接RFCOMM串口

1
2
3
 sudo rfcomm connect /dev/rfcomm0 00:14:01:22:14:49 1
Connected /dev/rfcomm0 to 00:14:01:22:14:49 on channel 1
Press CTRL-C for hangup
  • 运行上面的命令无错,就可以使用 minicom 连接它了,sudo minicom -o -b 115200 -D /dev/rfcomm0
  • 如果运行图形配置向导,要先运行blueman-applet再运行blueman-manager,就可以配置了.

蓝牙错误

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
$ systemctl status bluetooth
● bluetooth.service - Bluetooth service
Loaded: loaded (/lib/systemd/system/bluetooth.service; enabled; vendor preset: enabled)
Active: failed (Result: signal) since Wed 2018-06-20 14:54:03 CST; 5min ago
Docs: man:bluetoothd(8)
Main PID: 772 (code=killed, signal=KILL)
Status: "Running"
michael@debian:/etc/init.d$ sudo systemctl status bluetooth
● bluetooth.service - Bluetooth service
Loaded: loaded (/lib/systemd/system/bluetooth.service; enabled; vendor preset: enabled)
Active: failed (Result: signal) since Wed 2018-06-20 14:54:03 CST; 5min ago
Docs: man:bluetoothd(8)
Main PID: 772 (code=killed, signal=KILL)
Status: "Running"

Jun 20 14:41:29 debian bluetoothd[772]: Failed to obtain handles for "Service Changed" characteristic
Jun 20 14:41:29 debian bluetoothd[772]: Sap driver initialization failed.
Jun 20 14:41:29 debian bluetoothd[772]: sap-server: Operation not permitted (1)
Jun 20 14:41:29 debian bluetoothd[772]: Endpoint registered: sender=:1.67 path=/MediaEndpoint/A2DPSource
Jun 20 14:41:29 debian bluetoothd[772]: Endpoint registered: sender=:1.67 path=/MediaEndpoint/A2DPSink
Jun 20 14:42:00 debian bluetoothd[772]: vendor 0x0 product: 0x0
Jun 20 14:43:36 debian bluetoothd[772]: vendor 0x0 product: 0x0
Jun 20 14:54:03 debian systemd[1]: bluetooth.service: Main process exited, code=killed, status=9/KILL
Jun 20 14:54:03 debian systemd[1]: bluetooth.service: Unit entered failed state.
Jun 20 14:54:03 debian systemd[1]: bluetooth.service: Failed with result 'signal'.

无法连接蓝牙音响

1
2
3
4
sudo apt install pulseaudio-module-bluetooth
pulseaudio -k
pulseaudio --start
sudo pactl load-module module-bluetooth-discover

Linux Systemctl启动服务超时设置

  • 下面用一个网络连接服务为示例,如果网络配置是 DHCP,刚好网络没有提供 DHCP 服务,这时该服务程序会重试(默认)两分钟才会报错,这里可以设置成5到10 秒,快速跳过该服务启动.
  • 在文件/lib/systemd/system/networking.service.d/network-pre.conf里添加下面两行:
1
2
[Service]
TimeoutStartSec=15
  • 可以使用下面的方式来定位整个启动时的用时。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
~$ systemd-analyze critical-chain
The time when unit became active or started is printed after the "@" character.
The time the unit took to start is printed after the "+" character.

graphical.target @1min 33.969s
└─multi-user.target @1min 33.969s
└─getty.target @1min 33.969s
└─getty@tty1.service @1min 33.968s
└─rc-local.service @1min 32.592s +1.374s
└─network-online.target @1min 32.549s
└─network.target @1min 32.549s
└─networking.service @2.374s +898ms
└─systemd-sysctl.service @2.355s +4ms
└─systemd-modules-load.service @246ms +2.089s
└─systemd-journald.socket @227ms
└─-.mount @179ms
└─-.slice @179ms

  • 如上所示,需要查一下/etc/network/interfaces里的配置,rc-local里的配置,是什么原因导致在这里启动耗时。

安装 IBus 输入法框架

  • IBus("Intelligent Input Bus")是一个输入法框架,一个输入非英语字符的系统.IBus的功能与SCIMUIM类似.在Debian下面有一个问题,如果不安装ibus-gtk3,ibus-gtk就会出现wxWidgets框架下的文本输入框无法使用Back Space,Arrown keys这些按键.安装指导
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
~$ dpkg -l | grep "ibus"
ii gir1.2-ibus-1.0:amd64 1.5.14-3 amd64 Intelligent Input Bus - introspection data
ii ibus 1.5.14-3 amd64 Intelligent Input Bus - core
ii ibus-gtk:amd64 1.5.14-3 amd64 Intelligent Input Bus - GTK+2 support
ii ibus-gtk3:amd64 1.5.14-3 amd64 Intelligent Input Bus - GTK+3 support
ii ibus-qt4 1.3.3-1 amd64 qt-immodule for ibus (QT4) (plugin)
ii ibus-table 1.9.14-3 all table engine for IBus
ii ibus-table-wubi 1.8.2-2 all ibus-table input method: Wubi
ii libgusb-dev 0.2.9-1+b1 amd64 GLib wrapper around libusb1 - development files
ii libgusb2:amd64 0.2.9-1+b1 amd64 GLib wrapper around libusb1
ii libhidapi-libusb0:amd64 0.8.0~rc1+git20140818.d17db57+dfsg-1 amd64 Multi-Platform library for communication with HID devices (libusb backend)
ii libibus-1.0-5:amd64 1.5.14-3 amd64 Intelligent Input Bus - shared library
ii libibus-1.0-dev:amd64 1.5.14-3 amd64 Intelligent Input Bus - development file
ii libibus-qt1 1.3.3-1 amd64 qt-immodule for ibus (QT4) (library)

~$ cat ~/.xinitrc
export GTK_IM_MODULE=ibus
export QT_IM_MODULE=ibus
export XMODIFIERS="@im=ibus"
  • 如果要配置locales,方法如下:
    1
    2
    3
    ~$ sudo apt-get install locales
    ~$ sudo dpkg-reconfigure locales
    ~$ sudo locale-gen

BOSSAC 烧写 Arduino-DUE 无法找到设备

1
2
3
4
5
6
./bossac -i -d --port=ttyACM0 -U false -e -w -v -b ./nuttx.bin -R

Set binary mode
Send auto-baud
Set binary mode
No device found on /dev/ttyACM0

处理方法

1
stty -F /dev/ttyACM0 speed 1200 cs8 -cstopb -parenb

Virt-Manager 虚拟机无法启动

  • 有时KVM框架虚拟机系统无法启动,错误如下:

    1
    2
    3
    4
    Error starting domain: Requested operation is not valid: network 'default' is not active

    Traceback (most recent call last):
    [...]
  • 通过如下命令启动网络即可.

1
2
3
4
5
6
7
~$ sudo virsh net-list --all
Name State Autostart Persistent
----------------------------------------------------------
default inactive no yes

~$ sudo virsh net-start default
Network default started

Virt-Manager 使用 OpenvSwitch 交换机

  • 使用 OpenvSwitch 的网络服务启动会有延迟一分钟左右.

安装 OpenvSwitch

1
2
3
~$ dpkg -l | grep "openvswitch"
ii openvswitch-common 2.6.2~pre+git20161223-3 amd64 Open vSwitch common components
ii openvswitch-switch 2.6.2~pre+git20161223-3 amd64 Open vSwitch switch implementations

创建桥接端口

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
~$ sudo ovs-vsctl add-br ovsbr0
~$ sudo ovs-vsctl add-port ovsbr0 eth0
~$ sudo ovs-vsctl show
dbeec575-be91-48b6-bffa-0894b0fa24dd
Bridge "ovsbr0"
Port "ovsbr0"
Interface "ovsbr0"
type: internal
Port "vnet0"
Interface "vnet0"
Port "eth0"
Interface "eth0"
ovs_version: "2.6.2"

~$ sudo ip addr del 192.168.1.100 dev eth0
~$ sudo ip addr add 192.168.1.100 dev ovsbr0

修改网卡启动配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
~$ cat /etc/network/interfaces
[...]

auto eth0
iface eth0 inet manual
up ip link set dev $IFACE up
down ip link set dev $IFACE down


auto ovsbr0
iface ovsbr0 inet static
up ip link set dev $IFACE up
down ip link set dev $IFACE down
address 192.168.1.100
netmask 255.255.255.0
gateway 192.168.1.1

[...]

创建 Virt-Manager 桥接网络

  • Network_bridge
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    ~$ sudo cp /etc/libvirt/qemu/networks/{default.xml,ovsbr0.xml}

    # 把内容修改成如下所示
    ~$ cat /etc/libvirt/qemu/networks/ovsbr0.xml
    <!--
    WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
    OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
    virsh net-edit ovsbr0
    or other application using the libvirt API.
    -->

    <network>
    <name>ovsbr0</name>
    <uuid>fc0b3bb1-c273-49dc-b469-153dc4c452e2</uuid>
    <forward mode='bridge'/>
    <bridge name='ovsbr0'/>
    <virtualport type='openvswitch'/>
    </network>

    ~$ sudo virsh net-define /etc/libvirt/qemu/networks/ovsbr0.xml
    ~$ sudo virsh net-create /etc/libvirt/qemu/networks/ovsbr0.xml

    ~$ sudo virsh net-list --all
    Name State Autostart Persistent
    ----------------------------------------------------------
    default active yes yes
    ovsbr0 inactive no yes

    ~$ sudo virsh net-autostart ovsbr0

设置虚拟机的网络

  • 现在 KVM 虚拟机就有default,ovsbr0两个网络,添加一个连接到OpenvSwitch上的网卡如下图所示:
    ovsbr0

创建Debian Live USB系统盘

  • 如果想要试用Linux,又不想安装它,可以把它烧写到U盘里,做为启动盘或者系统恢复盘.
  • Debian 国内镜像下载链接
  • 注意: 这里一定要选择live标志的 iso 镜像下载.在 windows 下面要先下载一个叫Win32DiskImager的烧写工具,Linux 就很简单使用下面命令就能完成:
1
2
3
4
5
6
$ ls  debian-live-9.4.0-amd64-mate.iso
debian-live-9.4.0-amd64-mate.iso
$ sudo dd if=debian-live-9.4.0-amd64-mate.iso of=/dev/sdc1
4002944+0 records in
4002944+0 records out
2049507328 bytes (2.0 GB, 1.9 GiB) copied, 7.39192 s, 277 MB/s
  • 严重注意 这里的/dev/sdc1一要是你想要格式华的U盘,如果这里是当前硬盘,或者是有重要资料的盘符,就会直接覆盖它,找不回来.注意,注意,注意

开启Debian Hibernate功能

  • 有时工作进行到一个阶段,不想系统重启之后,再费时间来打开与配置环境.开启系统休眠功能非常有用.先要确保有一个swap分区,最好是两倍与系统内存的大小.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    ~$ sudo apt-get install acpi-support pm-utils
    ~$ sudo mkswap /dev/sdb1
    ~$ sudo swapon /dev/sdb1
    # 确保下面这两个文件里有这些行.
    ~$ sudo cat /etc/fstab | grep "swap"
    /dev/sdb1 none swap pri=1 0 0

    # 如果是在本机为其它机器编译内核与initramfs的,需要把resume=删除后,再make install,否则到目标机器上会panic.
    ~$ blkid
    [...]
    /dev/sdb3: UUID="ae170b9d-832a-46a7-beca-5fb7c9329b1b" TYPE="swap" PARTUUID="6daebe63-03"
    ~$ sudo cat /etc/default/grub | grep "resume"
    GRUB_CMDLINE_LINUX="resume=UUID=ae170b9d-832a-46a7-beca-5fb7c9329b1b"

    ~$cat /etc/initramfs-tools/conf.d/resume
    RESUME=UUID=ae170b9d-832a-46a7-beca-5fb7c9329b1b


    #这一操作很重要.
    ~$ update-initramfs -u -k `uname -r` ## 更新当前内核的启动镜像.
    ~$ sudo update-grub

    ~$ cat /proc/acpi/wakeup
    Device S-state Status Sysfs node
    PB31 S4 *disabled pci:0000:00:03.1
    PB32 S4 *disabled
    PB33 S4 *disabled pci:0000:00:03.3
    PB34 S4 *disabled
    SBAZ S4 *disabled pci:0000:00:14.2
    PS2K S3 *disabled
    PS2M S3 *disabled
    ECIR S4 *disabled
    P0PC S4 *disabled pci:0000:00:14.4
    OHC1 S4 *disabled pci:0000:00:12.0
    EHC1 S4 *disabled pci:0000:00:12.2
    OHC2 S4 *disabled pci:0000:00:13.0
    EHC2 S4 *disabled pci:0000:00:13.2
    OHC3 S4 *disabled
    EHC3 S4 *disabled
    OHC4 S4 *disabled pci:0000:00:14.5
    XHC0 S4 *disabled pci:0000:00:10.0
    XHC1 S4 *disabled pci:0000:00:10.1
    PE20 S4 *disabled
    PE21 S4 *disabled
    PE22 S4 *disabled
    PE23 S4 *disabled
    PWRB S3 *enabled platform:PNP0C0C:00

    # 上面是一些如何唤醒休眠的设备, 休眠中是可以通过移动鼠标或者键盘来唤醒,但是我这里把它们都关闭,让它只能通过电源按钮(PWRB)唤醒系统.

    ~$ grep "echo" /etc/rc.local
    [...]
    for i in OHC1 EHC1 OHC2 EHC2 OHC4 XHC0 XHC1;
    do
    cat /proc/acpi/wakeup | grep "$i" | grep "enabled" && echo "$i" > /proc/acpi/wakeup
    done
    [...]
  • 测试休眠功能echo disk > /sys/power/state,或者运行pm-hibernate进行休眠.

  • 通过上面设置,系统休眠与唤醒都很符合我的预期.还有一种方法是使用一个交换文件,不需要一个独立的分区,请参考这里

  • 如果上述设置能休眠不能关机,就是休眠后马上又重启了,参照这里需要如下设置:

1
2
3
4
/etc/systemd/sleep.conf.d/hibernatemode.conf

[Sleep]
HibernateMode=shutdown
  • 如果不能使用界的System->Shut Down->Hibernate的进行休眠,但是可以直接使用sudo pm-hibernate.需要重装下面的包试试.

    1
    ~$ apt-get install --reinstall pm-utils mate-power-manager-common hibernate upower
  • 如果休眠后恢复系统不能使用usb wifi(rtl8812au) 网卡.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    ~$ sudo  lshw -C network | grep "wlan" -A +3 | grep "driver"
    configuration: broadcast=yes driver=rtl8812au ip=192.168.5.190 multicast=yes wireless=IEEE 802.11
    # 创建如下文件
    ~$ sudo cat /etc/pm/config.d/unload_modules
    SUSPEND_MODULES="$SUSPEND_MODULES rtl8812au"
    # 创建成功,要加上运行权限.
    ~$ sudo cat /etc/pm/sleep.d/10_resume_wifi
    #!/bin/sh

    case "${1}" in
    resume|thaw)
    nmcli r wifi off && nmcli r wifi on;
    esac

程序找不到链接库位置

  • Python ImportError,这种错误是因为在PYTHONPATH路径中找不相就的模块.

    1
    2
    3
    4
    5
    6
    7
    ~$ ./modes_rx
    Traceback (most recent call last):
    File "./modes_rx", line 27, in <module>
    import air_modes
    ImportError: No module named air_modes

    ~$ PYTHONPATH=../lib/python2.7/dist-packages ./modes_rx #指定的环境变量对了就可以了.
  • 本地库链接错误

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    ~$ ldd /usr/local/bin/rtl_tcp
    linux-vdso.so.1 (0x00007fff3d3eb000)
    librtlsdr.so.0 => not found  # 默认位置/lib/x86_64-linux-gnu找不该库.
    libusb-1.0.so.0 => /lib/x86_64-linux-gnu/libusb-1.0.so.0 (0x00007f3d0b328000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3d0b108000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3d0ad68000)
    libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007f3d0b6d8000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f3d0b548000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f3d0ab60000)

    ~$ ls /usr/local/lib/librtlsdr.so.0  # 这个位置有这个库
    /usr/local/lib/librtlsdr.so.0
  • 上面的错误有两种方法可以解决:

    • /usr/local/lib加入到/etc/ld.so.conf里,再执行一次ldconfig就可以链接这个库了.
    • 使用LD_LIBRARY_PATH环境变量指位置,这样运行:
      1
      2
      3
      4
      5
      6
      7
      8
      ~$ LD_LIBRARY_PATH=/usr/local/lib  rtl_tcp
      Found 1 device(s).
      Found Rafael Micro R820T tuner
      Using ezcap USB 2.0 DVB-T/DAB/FM dongle
      Tuned to 100000000 Hz.
      listening...
      Use the device argument 'rtl_tcp=127.0.0.1:1234' in OsmoSDR (gr-osmosdr) source
      to receive samples in GRC and control rtl_tcp parameters (frequency, gain, ...).

在内核树里编译单个模块

  • 比如,上面缺少一个rfkill的模块,现在可以在不重编整个内核的前提下,可以单独编译这个内核.
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    ~$ cd /lib/modules/$(uname -r)/build/net/rfkill
    ~$ sudo make -C /lib/modules/$(uname -r)/build M=`pwd` modules # 要先在.config 把这一个模块选中为`编译`状态.
    make: Entering directory '/usr/src/linux-4.14.77'
    CC [M] /lib/modules/4.14.77-20181016-lcy-amd64/build/net/rfkill/core.o
    LD [M] /lib/modules/4.14.77-20181016-lcy-amd64/build/net/rfkill/rfkill.o
    Building modules, stage 2.
    MODPOST 1 modules
    CC /lib/modules/4.14.77-20181016-lcy-amd64/build/net/rfkill/rfkill.mod.o
    LD [M] /lib/modules/4.14.77-20181016-lcy-amd64/build/net/rfkill/rfkill.ko
    make: Leaving directory '/usr/src/linux-4.14.77'
    ~$ sudo mkdir /lib/modules/$(uname -r)/kernel/net/rfkill
    ~$ sudo cp rfkill.ko /lib/modules/$(uname -r)/kernel/net/rfkill/
    ~$ sudo depmod -a # 计算依赖关系
    ~$ modprobe -v rfkill # 加载模块.
  • 如果每次重启,某一个网块或蓝牙都是Soft Blocked的话,尝试一下用命令nmcli r wifi on.

编译carl9170出错问题

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
michael@debian:/usr/src/linux-5.5.11/drivers/net/wireless/ath/carl9170$ sudo make -C /lib/modules/$(uname -r)/build  M=`pwd`  modules
make: Entering directory '/usr/src/linux-5.5.11'
[....]
Building modules, stage 2.
MODPOST 1 modules
ERROR: "ath_regd_init" [/usr/src/linux-5.5.11/drivers/net/wireless/ath/carl9170/carl9170.ko] undefined!
ERROR: "__ieee80211_get_assoc_led_name" [/usr/src/linux-5.5.11/drivers/net/wireless/ath/carl9170/carl9170.ko] undefined!
ERROR: "led_classdev_register_ext" [/usr/src/linux-5.5.11/drivers/net/wireless/ath/carl9170/carl9170.ko] undefined!
ERROR: "ath_is_mybeacon" [/usr/src/linux-5.5.11/drivers/net/wireless/ath/carl9170/carl9170.ko] undefined!
ERROR: "ath_is_world_regd" [/usr/src/linux-5.5.11/drivers/net/wireless/ath/carl9170/carl9170.ko] undefined!
ERROR: "ath_reg_notifier_apply" [/usr/src/linux-5.5.11/drivers/net/wireless/ath/carl9170/carl9170.ko] undefined!
ERROR: "ath_regd_get_band_ctl" [/usr/src/linux-5.5.11/drivers/net/wireless/ath/carl9170/carl9170.ko] undefined!
ERROR: "__ieee80211_get_tx_led_name" [/usr/src/linux-5.5.11/drivers/net/wireless/ath/carl9170/carl9170.ko] undefined!
ERROR: "led_classdev_unregister" [/usr/src/linux-5.5.11/drivers/net/wireless/ath/carl9170/carl9170.ko] undefined!
make[1]: *** [scripts/Makefile.modpost:94: __modpost] Error 1
make: *** [Makefile:1607: modules] Error 2
make: Leaving directory '/usr/src/linux-5.5.11'
  • 像上面错误,后续不能在内核树编译驱动模块,因为选中该驱动中的子选项SoftLED Support(ONFIG_CARL9170_LEDS)是不支持单独编译成.ko或者包含在这个.ko里,它必须编译进内核.所以碰到这种问题,只能重新编译整个内核.

创建大于 2TB 的分区

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
~$parted /dev/sda
WARNING: You are not superuser. Watch out for permissions.
GNU Parted 3.2
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Model: ATA ST2000DM008-2FR1 (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags
(parted) mklabel gpt
(parted) print
Model: ATA ST2000DM008-2FR1 (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags

(parted) mkpart primary 0GB 2000GB

(parted) print
Model: ATA ST2000DM008-2FR1 (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags
1 1049kB 2000GB 2000GB primary
(parted) quit
~$ sudo fdisk -l /dev/sda
Disk /dev/sda: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 224F4012-9C49-4331-87F5-77B5CF6C28F5

Device Start End Sectors Size Type
/dev/sda1 2048 3907028991 3907026944 1.8T Linux filesystem


~$ sudo mkfs.ext4 /dev/sda1
mke2fs 1.43.4 (31-Jan-2017)
/dev/sda1 contains a ext3 file system
created on Thu Aug 30 09:33:27 2018
Proceed anyway? (y,N) y

~$ sudo blkid
/dev/sda1: UUID="3d6c0885-cacf-41f8-911d-cdf4bd666a49" TYPE="ext4" PARTLABEL="primary" PARTUUID="daeb01c4-ceb5-4227-80d8-d70bfc266317"
~$ cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
UUID=3d6c0885-cacf-41f8-911d-cdf4bd666a49 /data ext4 defaults 0 2

Linux 下修复(屏蔽)磁盘坏块

SmartCtl

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
~$ sudo  smartctl -H /dev/sdaX
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

~$ ~$ sudo smartctl -a /dev/sda | grep '^ID#' -A 27
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 077 049 006 Pre-fail Always - 55337314
3 Spin_Up_Time 0x0003 098 096 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 099 099 020 Old_age Always - 1081
5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0
7 Seek_Error_Rate 0x000f 084 060 045 Pre-fail Always - 248843209
9 Power_On_Hours 0x0032 092 092 000 Old_age Always - 7744 (71 178 0)
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 099 099 020 Old_age Always - 1081
183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0
184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0
187 Reported_Uncorrect 0x0032 084 084 000 Old_age Always - 16
188 Command_Timeout 0x0032 099 099 000 Old_age Always - 5 5 14
189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0
190 Airflow_Temperature_Cel 0x0022 062 053 040 Old_age Always - 38 (Min/Max 30/39)
191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 75
193 Load_Cycle_Count 0x0032 099 099 000 Old_age Always - 3229
194 Temperature_Celsius 0x0022 038 047 000 Old_age Always - 38 (0 13 0 0 0)
195 Hardware_ECC_Recovered 0x001a 077 064 000 Old_age Always - 55337314
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 8
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 8
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 6259h+10m+59.473s
241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 15449989789
242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 33054634173
[...]

# 我这边是希捷硬盘ST4000DM000
SMART Error Log Version: 1
ATA Error Count: 16 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 16 occurred at disk power-on lifetime: 7630 hours (317 days + 22 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 d5 08 ff ff ff 4f 00 5d+06:43:53.155 READ DMA EXT
25 d5 08 ff ff ff 4f 00 5d+06:43:53.114 READ DMA EXT
b0 d5 01 01 4f c2 00 00 5d+06:40:32.613 SMART READ LOG
b0 d5 01 06 4f c2 00 00 5d+06:40:32.545 SMART READ LOG
b0 d0 01 00 4f c2 00 00 5d+06:40:32.455 SMART READ DATA
  • 上面分析结果Current_Pending_Sector显示代表有8处坏块.

使用Short Test

1
2
3
~$ smartctl -t short /dev/sda
# 查看测试结果
~$ smartctl -l selftest /dev/sda

使用Long Test

  • 如果短测试没有发现问题,可能要使用长测试,耗时根据硬盘容量决定,ST4000DM000使用了大概480分钟.
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    ~$ smartctl -t long /dev/sda
    # 查看测试结果
    ~$ smartctl -x /dev/sda
    [...]
    SMART Extended Self-test Log Version: 1 (1 sectors)
    Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
    # 1 Short offline Completed without error 00% 7757 -
    # 2 Short offline Completed without error 00% 7757 -
    # 3 Extended offline Completed without error 00% 7752 -
    # 4 Short offline Completed without error 00% 7744 -
    # 5 Short offline Completed: read failure 90% 7744 5341719016
    # 6 Short offline Completed without error 00% 7744 -
    # 7 Extended offline Completed without error 00% 3769 -
    # 8 Extended offline Completed without error 00% 3746 -
    1 of 1 failed self-tests are outdated by newer successful extended offline self-test # 3
    [...]

NVMe的固态硬盘

防止/etc/resolve.conf 被 DHCP 自动修改

1
2
3
4
5
6
7
~# cat /etc/NetworkManager/NetworkManager.conf
[main]
plugins=ifupdown,keyfile
dns=none

[ifupdown]
managed=true
  • 或者用chattr +i /etc/resolve.conf锁定修改它.

apt-get使用 Socks5代理

1
2
3
4
5
6
7
~$ sudo apt-get install proxychains

~$ cat /etc/proxychains.conf
socks5 127.0.0.1 1080

~$ sudo proxychains apt-get update
~$ sudo proxychains apt-get dist-upgrade -y

Git使用问题

  • 统计工程代码数量.
    1
    ~$ git log --since="2018-03-01" --before="2019-01-09" --author="username" --pretty=tformat: --numstat | awk '{ add += $1; subs += $2; loc += $1 - $2 } END { printf "added lines: %s, removed lines: %s, total lines: %s\n", add, subs, loc }'

脚本配置git Socks5代理

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
~$ cat /usr/local/bin/gitproxy
#!/bin/bash
case $1 in

on)
git config --global http.proxy 'socks5://127.0.0.1:1080'
git config --global https.proxy 'socks5://127.0.0.1:1080'
;;

off)
git config --global --unset http.proxy
git config --global --unset https.proxy
;;

status)
git config --get http.proxy
git config --get https.proxy
;;
esac
exit 0

# 打开
~$ gitproxy on

# 关闭
~$ gitproxy off

Git 错误 error: refs/remotes/origin/rpi-4.19.y does not point to a valid object!

使用Intel AX200网卡

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
~$ hwinfo --netcard --bluetooth | grep -v "Config S"
06: PCI 300.0: 0282 WLAN controller
[Created at pci.386]
Unique ID: y9sn.yLXDVFQHSM7
Parent ID: W_nd.YYC8aCLnN82
SysFS ID: /devices/pci0000:00/0000:00:03.4/0000:03:00.0
SysFS BusID: 0000:03:00.0
Hardware Class: network
Model: "Intel WLAN controller"
Vendor: pci 0x8086 "Intel Corporation"
Device: pci 0x2723
SubVendor: pci 0x8086 "Intel Corporation"
SubDevice: pci 0x008c
Revision: 0x1a
Driver: "iwlwifi"
Driver Modules: "iwlwifi"
Device File: wlan0
Features: WLAN
Memory Range: 0xfe800000-0xfe803fff (rw,non-prefetchable)
IRQ: 50 (no events)
HW Address: 34:13:e8:b7:1d:76
Permanent HW Address: 34:13:e8:b7:1d:76
Link detected: yes
WLAN channels: 1 2 3 4 5 6 7 8 9 10 11 12 13 36 40 44 48 52 56 60 64 100 104 108 112 116 120 124 128 132 136 140
WLAN frequencies: 2.412 2.417 2.422 2.427 2.432 2.437 2.442 2.447 2.452 2.457 2.462 2.467 2.472 5.18 5.2 5.22 5.24 5.26 5.28 5.3 5.32 5.5 5.52 5.54 5.56 5.58 5.6 5.62 5.64 5.66 5.68 5.7
WLAN encryption modes: WEP40 WEP104 TKIP CCMP
WLAN authentication modes: open sharedkey wpa-psk wpa-eap
Module Alias: "pci:v00008086d00002723sv00008086sd0000008Cbc02sc80i00"
Driver Info #0:
Driver Status: iwlwifi is active
Driver Activation Cmd: "modprobe iwlwifi"
Attached to: #19 (PCI bridge)

44: USB 00.1: 11500 Bluetooth Device
[Created at usb.122]
Unique ID: Ue9G.qmlouffvxgF
Parent ID: 9Cfs.erpEvbsFWX1
SysFS ID: /devices/pci0000:00/0000:00:13.0/usb10/10-2/10-2:1.1
SysFS BusID: 10-2:1.1
Hardware Class: bluetooth
Model: "Intel Bluetooth Device"
Hotplug: USB
Vendor: usb 0x8087 "Intel Corp."
Device: usb 0x0029
Revision: "0.01"
Driver: "btusb"
Driver Modules: "btusb"
Speed: 12 Mbps
Module Alias: "usb:v8087p0029d0001dcE0dsc01dp01icE0isc01ip01in01"
Driver Info #0:
Driver Status: btusb is active
Driver Activation Cmd: "modprobe btusb"
Attached to: #45 (Hub)

~# modinfo -p iwlwifi
debug:debug output mask (uint)
swcrypto:using crypto in software (default 0 [hardware]) (int)
11n_disable:disable 11n functionality, bitmap: 1: full, 2: disable agg TX, 4: disable agg RX, 8 enable agg TX (uint)
amsdu_size:amsdu size 0: 12K for multi Rx queue devices, 2K for AX210 devices, 4K for other devices 1:4K 2:8K 3:12K 4: 2K (default 0) (int)
fw_restart:restart firmware in case of error (default true) (bool)
antenna_coupling:specify antenna coupling in dB (default: 0 dB) (int)
nvm_file:NVM file name (charp)
uapsd_disable:disable U-APSD functionality bitmap 1: BSS 2: P2P Client (default: 3) (uint)
enable_ini:Enable debug INI TLV FW debug infrastructure (default: 0 (bool)
bt_coex_active:enable wifi/bt co-exist (default: enable) (bool)
led_mode:0=system default, 1=On(RF On)/Off(RF Off), 2=blinking, 3=Off (default: 0) (int)
power_save:enable WiFi power management (default: disable) (bool)
power_level:default power save level (range from 1 - 5, default: 1) (int)
fw_monitor:firmware monitor - to debug FW (default: false - needs lots of memory) (bool)
disable_11ac:Disable VHT capabilities (default: false) (bool)
remove_when_gone:Remove dev from PCIe bus if it is deemed inaccessible (default: false) (bool)
disable_11ax:Disable HE capabilities (default: false) (bool)

~$ modinfo iwlwifi | grep "firmware"
[....]
firmware: iwlwifi-8000C-36.ucode
firmware: iwlwifi-9260-th-b0-jf-b0-46.ucode
firmware: iwlwifi-9000-pu-b0-jf-b0-46.ucode
firmware: iwlwifi-ty-a0-gf-a0-52.ucode
firmware: iwlwifi-so-a0-gf-a0-52.ucode
firmware: iwlwifi-so-a0-hr-b0-52.ucode
firmware: iwlwifi-so-a0-jf-b0-52.ucode
firmware: iwlwifi-cc-a0-52.ucode
firmware: iwlwifi-QuQnj-b0-jf-b0-52.ucode
firmware: iwlwifi-QuZ-a0-jf-b0-52.ucode
firmware: iwlwifi-QuZ-a0-hr-b0-52.ucode
[...]

~$ tree /sys/class/net
/sys/class/net
├── eth0 -> ../../devices/pci0000:00/0000:00:03.3/0000:02:00.0/net/eth0
├── lo -> ../../devices/virtual/net/lo
└── wlan0 -> ../../devices/pci0000:00/0000:00:03.4/0000:03:00.0/net/wlan0
thermal thermal_zone0: failed to read out thermal zone (-61)
  • 测试不同的加载参数,下面参数等于echo "options iwlwifi 11n_disable=8" > /etc/modprobe.d/iwlwifi.conf.只是要重启生效

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    ~$ sudo rmmod iwlmvm
    ~$ sudo rmmod iwlwifi
    ~$ sudo modprobe iwlwifi 11n_disable=8

    # 对应的参数说明是

    ~$ modinfo iwlwifi
    [...]
    parm: swcrypto:using crypto in software (default 0 [hardware]) (int)
    parm: 11n_disable:disable 11n functionality, bitmap: 1: full, 2: disable agg TX, 4: disable agg RX, 8 enable agg TX (uint)
    parm: amsdu_size:amsdu size 0: 12K for multi Rx queue devices, 2K for AX210 devices, 4K for other devices 1:4K 2:8K 3:12K 4: 2K (default 0) (int)
    parm: fw_restart:restart firmware in case of error (default true) (bool)
    parm: antenna_coupling:specify antenna coupling in dB (default: 0 dB) (int)
    parm: nvm_file:NVM file name (charp)
    parm: uapsd_disable:disable U-APSD functionality bitmap 1: BSS 2: P2P Client (default: 3) (uint)
    parm: enable_ini:Enable debug INI TLV FW debug infrastructure (default: 0 (bool)
    parm: bt_coex_active:enable wifi/bt co-exist (default: enable) (bool)
    parm: led_mode:0=system default, 1=On(RF On)/Off(RF Off), 2=blinking, 3=Off (default: 0) (int)
    parm: power_save:enable WiFi power management (default: disable) (bool)
    parm: power_level:default power save level (range from 1 - 5, default: 1) (int)
    parm: fw_monitor:firmware monitor - to debug FW (default: false - needs lots of memory) (bool)
    parm: disable_11ac:Disable VHT capabilities (default: false) (bool)
    parm: remove_when_gone:Remove dev from PCIe bus if it is deemed inaccessible (default: false) (bool)
    parm: disable_11ax:Disable HE capabilities (default: false) (bool)

    # 查看当前驱动的参数
    ~# systool -m iwlwifi -av
    Module = "iwlwifi"

    Attributes:
    coresize = "282624"
    initsize = "0"
    initstate = "live"
    refcnt = "1"
    srcversion = "4E773ECBB9682B55D952A3C"
    taint = ""
    uevent = <store method only>

    Parameters:
    11n_disable = "0"
    amsdu_size = "0"
    antenna_coupling = "0"
    bt_coex_active = "Y"
    debug = "0"
    disable_11ac = "N"
    disable_11ax = "N"
    enable_ini = "N"
    fw_monitor = "N"
    fw_restart = "Y"
    led_mode = "0"
    nvm_file = "(null)"
    power_level = "0"
    power_save = "N"
    remove_when_gone = "N"
    swcrypto = "0"
    uapsd_disable = "3"
    [....]
    # 对应映射到系统的位置.
    ~$ ls /sys/module/iwlwifi/parameters/
    11n_disable antenna_coupling disable_11ac enable_ini fw_restart nvm_file power_save swcrypto
    amsdu_size bt_coex_active disable_11ax fw_monitor led_mode power_level remove_when_gone uapsd_disable

  • 最终参照这里,开启优化设置如下:

1
2
3
4
5
6
7
8
9
~$  cat /etc/modprobe.d/iwlwifi.conf
options iwlwifi 11n_disable=8
options iwlwifi swcrypto=1
options iwlwifi bt_coex_active=0
options iwlwifi power_save=0
options iwlwifi d0i3_disable=0
options iwlwifi amsdu_size=3
options iwlmvm power_scheme=3
options iwldvm force_cam=0
  • hcitool 连接蓝牙
1
2
3
4
5
6
7
8
9
10
~$ hcitool scan
Scanning ...
00:14:01:22:14:49 lcy
B8:AD:3E:91:70:02 LG HBS500
98:D3:36:00:CC:BF lcy_HC-05

~$ hcitool cc B8:AD:3E:91:70:02
Can't create connection: Operation not permitted

~$ sudo setcap cap_net_raw+eip `which hcitool`
  • 无法检测到蓝牙控制器
1
2
~$ bluetoothctl show
No default controller available
  • 尝试重装一下它的驱动
1
2
3
4
5
~# rmmod btusb
~# rmmod btintel

~# modprobe btintel
~# modprobe btusb

查看WIFI的地区代码.

  • iwconfig wlan0 txpower 30mW - not working
  • How to create custom Linux Wi-Fi regulatory database to unlock 30db/1000mWDFS是为了使无线产品主动探测军方使用的频率,并主动选择另一个频率,以避开军方频率.所以上面那个链接说,在美国解除限制是违法.
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    ~$ regdbdump /lib/crda/regulatory.bin
    ~$ iw reg get
    phy#0 (self-managed)
    country CN: DFS-UNSET
    (2402 - 2437 @ 40), (6, 22), (N/A), AUTO-BW, NO-HT40MINUS, NO-80MHZ, NO-160MHZ
    (2422 - 2462 @ 40), (6, 22), (N/A), AUTO-BW, NO-80MHZ, NO-160MHZ
    (2447 - 2482 @ 40), (6, 22), (N/A), AUTO-BW, NO-HT40PLUS, NO-80MHZ, NO-160MHZ
    (5170 - 5190 @ 160), (6, 22), (N/A), NO-OUTDOOR, AUTO-BW, IR-CONCURRENT, NO-HT40MINUS, PASSIVE-SCAN
    (5190 - 5210 @ 160), (6, 22), (N/A), NO-OUTDOOR, AUTO-BW, IR-CONCURRENT, NO-HT40PLUS, PASSIVE-SCAN
    (5210 - 5230 @ 160), (6, 22), (N/A), NO-OUTDOOR, AUTO-BW, IR-CONCURRENT, NO-HT40MINUS, PASSIVE-SCAN
    (5230 - 5250 @ 160), (6, 22), (N/A), NO-OUTDOOR, AUTO-BW, IR-CONCURRENT, NO-HT40PLUS, PASSIVE-SCAN
    (5250 - 5270 @ 160), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40MINUS, PASSIVE-SCAN
    (5270 - 5290 @ 160), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40PLUS, PASSIVE-SCAN
    (5290 - 5310 @ 160), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40MINUS, PASSIVE-SCAN
    (5310 - 5330 @ 160), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40PLUS, PASSIVE-SCAN
    (5490 - 5510 @ 240), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40MINUS, PASSIVE-SCAN
    (5510 - 5530 @ 240), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40PLUS, PASSIVE-SCAN
    (5530 - 5550 @ 240), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40MINUS, PASSIVE-SCAN
    (5550 - 5570 @ 240), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40PLUS, PASSIVE-SCAN
    (5570 - 5590 @ 240), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40MINUS, PASSIVE-SCAN
    (5590 - 5610 @ 240), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40PLUS, PASSIVE-SCAN
    (5610 - 5630 @ 240), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40MINUS, PASSIVE-SCAN
    (5630 - 5650 @ 240), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40PLUS, PASSIVE-SCAN
    (5650 - 5670 @ 80), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40MINUS, NO-160MHZ, PASSIVE-SCAN
    (5670 - 5690 @ 80), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40PLUS, NO-160MHZ, PASSIVE-SCAN
    (5690 - 5710 @ 80), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40MINUS, NO-160MHZ, PASSIVE-SCAN
    (5710 - 5730 @ 80), (6, 22), (0 ms), DFS, AUTO-BW, NO-HT40PLUS, NO-160MHZ, PASSIVE-SCAN
    (5735 - 5755 @ 80), (6, 22), (N/A), AUTO-BW, IR-CONCURRENT, NO-HT40MINUS, NO-160MHZ, PASSIVE-SCAN
    (5755 - 5775 @ 80), (6, 22), (N/A), AUTO-BW, IR-CONCURRENT, NO-HT40PLUS, NO-160MHZ, PASSIVE-SCAN
    (5775 - 5795 @ 80), (6, 22), (N/A), AUTO-BW, IR-CONCURRENT, NO-HT40MINUS, NO-160MHZ, PASSIVE-SCAN
    (5795 - 5815 @ 80), (6, 22), (N/A), AUTO-BW, IR-CONCURRENT, NO-HT40PLUS, NO-160MHZ, PASSIVE-SCAN
    (5815 - 5835 @ 20), (6, 22), (N/A), AUTO-BW, IR-CONCURRENT, NO-HT40MINUS, NO-HT40PLUS, NO-80MHZ, NO-160MHZ, PASSIVE-SCAN
  • 也可以在/etc/default/crda设置区域代码.
1
2
3
dmesg | grep "cfg80211"
[ 6.756930] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[ 6.756935] cfg80211: failed to load regulatory.db

编译wireless-regdb

1
2
3
4
5
6
# python2 required
~$ pip install M2Crypto
~$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/sforshee/wireless-regdb.git

~$ cd wireless-regdb && make
~$ make install

编译crda

1
~$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/crda.git

创建Linux路由器(支持IPv6-PD)

  • 链接:

  • 一般来说,可以使用Linux来做一个完全路由(OpentWrt),这里讲的是如何在自己工作的台式电脑内安装路由(含WiFi)功能.一般是台机是只有一个以太网卡,外接一个USB Wifi网卡来做无线路由,当然也可以有这些组合:eth0(WAN)–>wlan0(LAN), eth0(WAN)–>eth1(LAN),wlan0(WAN)–>wlan1(lan). 一般卖的小家用路由器是:一个WAN口,外加4个交换机LAN口,可能还有一个Wifi功能.其实这些低成本的路由器只有一个物理网卡,使用VLAN把它分成两个子网(WAN,LAN),其实也就是单臂路由.如:eth0.0(WAN)-->br-lan(eth0.1,wlan)使用桥接(bridge)功能把wifi与4个LAN口交换机绑定成一个内局域网内.

  • 下面这里是把两个wifi组成一个无线路由,wlan0(WAN)--wlan1(LAN),wlan0是从连接上层路由(外部ISP)相当于是WAN,wlan1是内部子网相当于是LAN.之前一般配置是,开启内核转发net.ipv4.ip_forward = 1,再配置一个hostapd,dnsmasq就能对外提供一个简单在IPv4的路由服务了.

系统配置

1
2
3
4
5
6
7
8
9
10
11
12
~$ cat /etc/sysctl.conf
# ipv4转发功能,必须开启,才能让本地ipv4地址上网
net.ipv4.ip_forward=1
# 开启接的路由广告功能
net.ipv6.conf.all.accept_ra = 2
net.ipv6.conf.default.accept_ra = 2
net.ipv6.conf.wlan0.accept_ra = 2

# IPv6转发,在IPv6时代,不需要NAT转发,全局地址可以直接访问到对方.
net.ipv6.conf.wlan0.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.default.forwarding = 1

DHCP&DNS服务器

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
~$ sudo apt-get instal dnsmasq

# 具体配置说明可以使用 man dnsmasq查看.
~$ cat /etc/dnsmasq.conf # grep -v '^#\|^$' /etc/dnsmasq.conf
domain-needed
bogus-priv
server=2606:4700:4700::1001
server=8.8.8.8
no-resolv
server=192.168.1.1
interface=wlan1
except-interface=wlan0
listen-address=192.168.2.1
no-dhcp-interface=wlan0,lo
dhcp-range=192.168.2.50,192.168.2.150,12h
# 下面两行很重要,开启ipv6路由广告,指定接口提供PD.
enable-ra
dhcp-range=tag:wlan1,::1,ra-names,slaac,constructor:wlan1,64,12h
dhcp-ignore-names=tag:dhcp_bogus_hostname
dhcp-rapid-commit
domain=lan
server=/lan/
#log-facility=/var/log/dnsmasq.log

DHCP客户端

  • 如果只是ipv4路由服务,是不需安装下面这些DHCP客户端的.这里使用的是dhcpcd5,功能:DHCPv4, IPv6RA and DHCPv6 client with IPv4LL support.还有如:wide-dhcpv6-client,dibbler-client.这里是以dhcpcd5为示例,具体详细配置可以通过man dhcpcd.conf查看.

DHCPCD5

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
~$ sudo apt-get instal dhcpcd5

~$ cat /etc/dhcpcd.conf # 可以这样过滤注释 grep -v '^#\|^$' /etc/dhcpcd.conf
hostname
# logfile /var/log/dhcpcd.log 不指定它,就输出到/var/log/syslog
duid
noipv6rs # 先要全局禁止ipv6请求.
waitip 6
persistent
option rapid_commit
option domain_name_servers
option classless_static_routes
option interface_mtu
require dhcp_server_identifier
# debug
slaac private

# 这里开始,配置wlan0接口 WAN
interface wlan0
ipv6rs
#ia_na 1
ia_pd 1 wlan1/0 # 为 wlan1(LAN)请求 Prefix-delegated 地址.

~$ systemctl start dhcpcd

Wide-dhcpv6-client

  • wide-dhcpv6-client与dhcpcd二者不能同时运行.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    ~$ sudo apt-get install wide-dhcpv6-client
    ~$ cat /etc/wide-dhcpv6/dhcp6c.conf
    # Default dhpc6c configuration: it assumes the address is autoconfigured using
    # router advertisements.
    interface wlan0 {
    send ia-pd 0;
    send rapid-commit;
    request domain-name-servers;
    script "/sbin/dhcp6c-state";
    };

    id-assoc pd 0 {
    prefix-interface wlan1 {
    sla-id 0;
    sla-len 0;
    };
    };

    ~$ sudo /etc/init.d/wide-dhcpv6-client start
  • 启动/etc/init.d/dhcpcd restart,获取PD信息根据网络质量,可能会有不到十几秒的延时,现在查看接口会显示类似如下:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    ~$ ifconfig wlan1
    wlan1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
    inet 192.168.2.1 netmask 255.255.255.0 broadcast 192.168.2.255
    inet6 2409:8a55:xxxx:xxxx::1 prefixlen 64 scopeid 0x0<global> # 这里是通过wlan1从上游ISP获取的PD.
    inet6 fe80::705c:99ff:feda:9f46 prefixlen 64 scopeid 0x20<link>
    ether 72:5c:99:da:9f:46 txqueuelen 1000 (Ethernet)
    RX packets 174 bytes 21784 (21.2 KiB)
    RX errors 0 dropped 0 overruns 0 frame 0
    TX packets 401 bytes 76160 (74.3 KiB)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Wifi客户端

HOSTAPD

  • wiki/Hostapd

  • 不幸的,intel iwlwifi它在hostapd不支持AC 5G的热点功能。

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    ~$ sudo apt-get install hostapd
    ~$ cat /etc/hostapd/hostapd.conf
    interface=wlan1
    # bridge=br-lan
    driver=nl80211
    logger_syslog=-1
    logger_syslog_level=2
    logger_stdout=-1
    logger_stdout_level=2
    ctrl_interface=/var/run/hostapd
    ctrl_interface_group=0
    ssid=lcy
    country_code=US
    hw_mode=g # b,g,a三种模式,5G 是ac模式
    channel=1
    beacon_int=100
    dtim_period=2
    macaddr_acl=0
    auth_algs=1
    wmm_enabled=1
    wmm_ac_bk_cwmin=4
    wmm_ac_bk_cwmax=10
    wmm_ac_bk_aifs=7
    wmm_ac_bk_txop_limit=0
    wmm_ac_bk_acm=0
    wmm_ac_be_aifs=3
    wmm_ac_be_cwmin=4
    wmm_ac_be_cwmax=10
    wmm_ac_be_txop_limit=0
    wmm_ac_be_acm=0
    wmm_ac_vi_aifs=2
    wmm_ac_vi_cwmin=3
    wmm_ac_vi_cwmax=4
    wmm_ac_vi_txop_limit=94
    wmm_ac_vi_acm=0
    wmm_ac_vo_aifs=2
    wmm_ac_vo_cwmin=2
    wmm_ac_vo_cwmax=3
    wmm_ac_vo_txop_limit=47
    wmm_ac_vo_acm=0
    # ieee80211n=1 # 开启11N模式
    # ht_capab=[HT40+][SHORT-GI-20][SHORT-GI-40]
    wpa=2
    wpa_psk_file=/etc/hostapd/hostapd.wpa_psk
    wpa_key_mgmt=WPA-PSK
    wpa_pairwise=TKIP
    rsn_pairwise=CCMP
    wpa_group_rekey=600

    ~$ systemctl start hostapd

WPASUPPLICANT

1
2
3
4
5
6
7
8
9
10
~$ sudo apt-get install wpasupplicant
~$ cat /etc/wpa_supplicant/lcy_wifi.conf
network={
ssid="lcy"
mode=2 # 2为AP模式.
#psk="xxxxxx"
psk=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
}

~$ wpa_supplicant -B -Dnl80211 -iwlan1 -c/etc/wpa_supplicant/lcy_wifi.conf
  • 上面这些工具,都可以是运行为独立的服务,可以像要下面一样,通过网卡事件脚本来控制.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    ~$ cat /etc/network/interfaces

    auto wlan1
    allow-hotplug wlan1
    iface wlan1 inet static
    address 192.168.2.1
    netmask 255.255.255.0
    network 192.168.2.0
    #up ifconfig ovsbr0 down
    # 使用wpa_supplicant 开启AP模式,提供wifi访问.
    pre-up wpa_supplicant -B -Dnl80211 -i$IFACE -c/etc/wpa_supplicant/lcy_wifi.conf \
    -P/var/run/wpa_supplicant.$IFACE.pid
    # 在这里 hostapd与wpa_supplicant二选一提供AP认证服务
    #post-up /usr/sbin/hostapd -B -P/var/run/hostapd.$IFACE.pid /etc/hostapd/hostapd.conf
    post-up /usr/sbin/dnsmasq --conf-file=/etc/dnsmasq.conf \
    --pid-file=/var/run/dnsmasq.$IFACE.pid
    post-up /etc/init.d/dhcpcd restart
    wait-delay 15
    pre-down cat /var/run/dnsmasq.$IFACE.pid | xargs kill
    pre-down cat /var/run/wpa_supplicant.$IFACE.pid | xargs kill
    #pre-down cat /var/run/hostapd.$IFACE.pid | xargs kill

    auto wlan0
    iface wlan0 inet static
    #up ip link set dev ovsbr0 down
    #up ifconfig ovs-system down
    #up ifconfig ovsbr0 down
    address 192.168.1.100
    netmask 255.255.255.0
    network 192.168.1.0
    gateway 192.168.1.1
    # 这里也很重要,WAN口需要固定一个MAC地址,
    # pre-up ip link set dev $IFACE addr 34:13:xx:xx:xx:76
    # 使用wpa_supplicant去连接上游的wifi.
    pre-up wpa_supplicant -B -Dnl80211 -i$IFACE -c/etc/wpa_supplicant/wpa_supplicant.conf \
    -P/var/run/wpa_supplicant.$IFACE.pid
    post-up /etc/init.d/dhcpcd start # 简单可以直接 dhcpcd $IFACE
    wait-delay 15
    post-down /etc/init.d/dhcpcd stop
    pre-down cat /var/run/wpa_supplicant.$IFACE.pid | xargs kill
  • 也可以使用下面wpa_supplicant方式连接认证wifi.用wpa_passphrase生成要连接的ssid与密钥.

    1
    2
    3
    4
    5
    6
    ~$ wpa_passphrase my_ssid 12345678
    network={
    ssid="my_ssid"
    #psk="12345678"
    psk=b63bb73550b08933c709bbff538af08d29024cbf0790981c116a3210391fe2d0
    }
  • 使用systemd控制

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
~$ cat /etc/systemd/system/network-wireless@.service:
[Unit]
Description=Wireless network connectivity (%i)
Wants=network.target
Before=network.target
BindsTo=sys-subsystem-net-devices-%i.device
After=sys-subsystem-net-devices-%i.device

[Service]
Type=oneshot
RemainAfterExit=yes

ExecStart=/usr/sbin/ip link set dev %i up
ExecStart=/usr/sbin/wpa_supplicant -B -i %i -c /etc/wpa_supplicant/wpa_supplicant.conf
#ExecStart=/usr/sbin/dhclient %i

ExecStop=/usr/sbin/ip link set dev %i down

[Install]
WantedBy=multi-user.target


~$ ln -s /etc/systemd/system/network-wireless@.service \
/etc/systemd/system/multi-user.target.wants/network-wireless@wlan0.service


~$ systemctl daemon-reload
~$ systemctl start network-wireless@wlan0.service

  • 最终的生成的数据如下:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    ~$ cat /etc/wpa_supplicant/wpa_supplicant.conf
    ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
    update_config=1
    p2p_disabled=1
    network={
    ssid="my_ssid"
    #psk="12345678"
    psk=b63bb73550b08933c709bbff538af08d29024cbf0790981c116a3210391fe2d0
    #COUNTRY=US
    proto=RSN
    key_mgmt=WPA-PSK
    pairwise=CCMP TKIP
    group=CCMP TKIP
    }
  • 如上所示,在/etc/network/interfaces的接口内加入pre-up wpa_supplicant -B -Dnl80211 -i$IFACE -c/etc/wpa_supplicant/wpa_supplicant.conf方式连接.如果要查看一些调试信息,加上-dd命令行中.

  • 查看状态

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    ~$ wpa_cli  status
    Selected interface 'wlan0'
    bssid=22:76:93:00:xx:xx
    freq=5200
    ssid=my_ssid
    id=0
    mode=station
    pairwise_cipher=CCMP
    group_cipher=CCMP
    key_mgmt=WPA2-PSK
    wpa_state=COMPLETED
    ip_address=192.168.1.100
    address=44:af:28:32:xx:xx
    uuid=d52e74ae-991c-5d82-ab4f-80a9f2417de5
    ieee80211ac=1
  • 使用手机连接到该wifi热点上,从手机信息的菜单项查看,是否获取一个240x开头的IPv6地址.如果有,说明可以使用IPv6访问网络了,打开test-ipv6.com测试一下.

  • 如果LAN的客户端能够正常获取一个PD+MAC全局地址,但是不能访问IPv6网络.这就要仔细查看与调试各级路由了.我在此因为设置了动态MAC,所以造成的路由问题.一般这里上游路由表如下:

    1
    2
    3
    4
    5
    6
    ~$ ip -6 route
    # 上游路由取得的PD是 2409:xxxx:xxxx:4fc0::/60
    default from 2409:xxxx:xxxx:4fc0::/60 via fe80::e681:84ff:fe44:a40f dev pppoe-wan proto static metric 512 pref medium
    2409:8a55:2416:4fc0::/64 dev br-lan proto static metric 1024 pref medium
    # 这里有一条静态的路由,去到这一级的WAN(wlan0)口的链路地址 fe80::xxxx:xxxx:21ba:6d93,这一条路由可以修改,但是这个客户端断开后,它又会重写回去.
    2409:8a55:xxxx:xxxx::/64 via fe80::xxxx:xxxx:21ba:6d93 dev br-lan proto static metric 1024 pref medium
  • 刚好我又在这个Linux内安装了macchanger这个工具脚本,它链接网卡的启停脚本.所以本机系统每个网卡MAC会因为每次up|down后随机产生一个新的地址,所以依赖于MAC计算出来的本地链路地址也变了,对于LAN口(wlan1)的地址改变其实没有什么影响,但是WAN口(wlan0)的地址改变就会影响到上游路由表无法到更新后的链路地址.或者说:本地WAN(wlan0)的链路地址改变,需同步更改上游路由表,否则本级路由下LAN(wlan)不能正常访问网络.

1
2
3
4
5
6
7
8
9
10
11
/etc/network$ tree if-pre-up.d/
if-pre-up.d/
├── bridge -> /lib/bridge-utils/ifupdown.sh
├── ethtool
├── hostapd -> ../../hostapd/ifupdown.sh
├── macchanger -> ../../macchanger/ifupdown.sh
├── openvswitch -> /usr/share/openvswitch/scripts/ifupdown.sh
├── uml-utilities
├── vlan
├── wireless-tools
└── wpasupplicant -> ../../wpa_supplicant/ifupdown.sh
  • macchanger可以查看硬件真实的MAC,也可生成随机MAC并修改它.这里需要在WAN口(wlan0)固定硬件地址的随机更改.如果要防止它每次重启后随机变动.一种方法确保有下面配置:
    1
    2
    3
    4
    ~$ cat /etc/NetworkManager/NetworkManager.conf
    [....]
    [device]
    wifi.scan-rand-mac-address=no
  • 或者使用macchanger,ip/etc/network/interfaces的里脚本里固定它.修改硬件地址先要关闭硬件功能,有时运行了如:ifdown wlan0 或者 ifconfig wlan0 down,还是没有关闭硬件,导致无法修改MAC,此时肯定还有其守护程序在运行,如:wicd,可以运行systemctl stop wicd再尝试修改.

Wifi Direct(p2p mode)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
~$ cat /etc/wpa_supplicant/wpa_supplicant.conf
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
device_name=debian
device_type=2-0050F204-1
p2p_go_intent=15
p2p_go_ht40=1
#p2p_disabled=1
network={
ssid="OpenWrt-5G"
#psk="xxxxxxxxxxxxxxxxxx"
psk=d5c46754b57b318f213eae32eefa61a602f28df87af6f5fd83c5297b775b6b82

#COUNTRY=US
proto=RSN
key_mgmt=WPA-PSK
pairwise=CCMP TKIP
group=CCMP TKIP
}


~$ wpa_supplicant -B -Dnl80211 -i$IFACE -dd -c/etc/wpa_supplicant/wpa_supplicant.conf
[....]
Using existing control interface directory.
ctrl_interface_group=113 (from group name 'netdev')
P2P: Add operating class 81
P2P: Channels - hexdump(len=13): 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d
P2P: Own listen channel: 81:6
P2P: Random operating channel: 81:1
P2P: initialized
P2P: channels: 81:1,2,3,4,5,6,7,8,9,10,11,12,13
P2P: cli_channels:
p2p-dev-wlan0: Added interface p2p-dev-wlan0
p2p-dev-wlan0: State: INACTIVE -> DISCONNECTED
nl80211: Set p2p-dev-wlan0 operstate 0->0 (DORMANT)
netlink: Operstate: ifindex=0 linkmode=-1 (no change), operstate=5 (IF_OPER_DORMANT)
p2p-dev-wlan0: Determining shared radio frequencies (max len 2)
p2p-dev-wlan0: Shared frequencies (len=0): completed iteration
P2P: Add operating class 81
P2P: Channels - hexdump(len=13): 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d
P2P: Update channel list
P2P: channels: 81:1,2,3,4,5,6,7,8,9,10,11,12,13
P2P: cli_channels:
Daemonize..
sending commands to master dhcpcd process
sending commands to master dhcpcd process

~$ sudo wpa_cli -i p2p-dev-wlan0

> p2p_find 10
OK
<3>CTRL-EVENT-SCAN-STARTED
<3>CTRL-EVENT-SCAN-STARTED
<3>CTRL-EVENT-SCAN-STARTED
<3>CTRL-EVENT-SCAN-STARTED
<3>P2P-DEVICE-FOUND be:30:xx:xx:19:a1 p2p_dev_addr=be:30:xx:xx:19:a1 pri_dev_type=7-0050F204-1 name='客厅的小米电视' config_methods=0x188 dev_capab=0x24 group_capab=0x0 wfd_dev_info=0x00112b441132 vendor_elems=1 new=0
<3>CTRL-EVENT-SCAN-STARTED
<3>CTRL-EVENT-SCAN-STARTED
<3>CTRL-EVENT-SCAN-STARTED
<3>P2P-FIND-STOPPED
<3>P2P-DEVICE-LOST p2p_dev_addr=be:30:xx:xx:19:a1
p2p_peers
be:30:xx:xx:19:a1
> p2p_connect be:30:xx:xx:19:a1 pin intent 15
55577670
<3>P2P-GO-NEG-SUCCESS role=GO freq=2412 ht40=1 peer_dev=be:30:xx:xx:19:a1 peer_iface=be:30:xx:xx:99:a1 wps_method=Display
<3>P2P-GROUP-FORMATION-SUCCESS
<3>P2P-GROUP-STARTED p2p-wlan0-0 GO ssid="DIRECT-vV" freq=2412 passphrase="BqbrqD4l" go_dev_addr=44:af:28:32:4f:de
<3>AP-STA-CONNECTED be:30:xx:xx:99:a1 p2p_dev_addr=be:30:xx:xx:19:a1

VirtualBox虚拟机与主机的复制与粘贴

  • 一般来说,如果虚拟机(Guest)安装的是Windows系统,只要安装VBoxGuestAddions.iso里的驱动,再在菜单里选择Devices-->Shared Clipboard方式,就可以与主机(Host)之前的共享复制粘贴了.如果GuestLinux系统,安装完成VBoxGuestAddions.iso且设置了Shared Clipboard,然而不能复制的话,可以在Guest里,运行sudo VBoxClient --clipboard之后,再尝试上述操作.

VirtualBox在AMD Ryzen 7运行出现Oops

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
~$ dpkg -l | grep "virtualbox"
ii virtualbox 6.1.28-dfsg-1~fto11+1 amd64 x86 virtualization solution - base binaries
ii virtualbox-dkms 6.1.28-dfsg-1~fto11+1 amd64 x86 virtualization solution - kernel module sources for dkms
ii virtualbox-ext-pack 6.1.28-1~fto11+1 all extra capabilities for VirtualBox, downloader.
ii virtualbox-guest-utils 6.1.28-dfsg-1~fto11+1 amd64 x86 virtualization solution - non-X11 guest utilities
ii virtualbox-guest-x11 6.1.28-dfsg-1~fto11+1 amd64 x86 virtualization solution - X11 guest utilities
ii virtualbox-qt 6.1.28-dfsg-1~fto11+1 amd64 x86 virtualization solution - Qt based user interface

~$ dmesg
[...]
[ 16.592665] systemd-journald[405]: File /var/log/journal/b41279e62373e95aba7f576b5274305b/user-1000.journal corrupted or uncleanly shut down, renaming and replacing.
[ 127.593723] SUPR0GipMap: fGetGipCpu=0x3b
[ 128.190967] kernel tried to execute NX-protected page - exploit attempt? (uid: 1000)
[ 128.190973] BUG: unable to handle page fault for address: ffffb050839f7000
[ 128.190975] #PF: supervisor instruction fetch in kernel mode
[ 128.190976] #PF: error_code(0x0011) - permissions violation
[ 128.190978] PGD 800100000067 P4D 800100000067 PUD 80010006a067 PMD 8003265ad067 PTE 8000800105ad6161
[ 128.190981] Oops: 0011 [#1] PREEMPT SMP NOPTI
[ 128.190982] CPU: 14 PID: 5557 Comm: EMT-0 Tainted: P OE 5.15.14-20220112 #1
[ 128.190985] Hardware name: ASUS System Product Name/TUF GAMING B550M-PLUS (WI-FI), BIOS 1401 12/03/2020
[ 128.190987] RIP: 0010:0xffffb050839f7000
[ 128.190989] Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[ 128.190990] RSP: 0018:ffffb0508388fe08 EFLAGS: 00010283
[ 128.190992] RAX: ffffb050839120f0 RBX: ffffb05083a8f010 RCX: ffffb05083891000
[ 128.190993] RDX: ffffb05083911370 RSI: 0000000000000000 RDI: ffff8d0d83852790
[ 128.190994] RBP: ffffb0508388fe98 R08: 0000000000000001 R09: ffffedc080000000
[ 128.190995] R10: ffffffffc306a6e0 R11: 00008003265ae007 R12: 0000000000000024
[ 128.190996] R13: ffffffffc306a6e0 R14: 0000000000000000 R15: ffff8d0d83852790
[ 128.190997] FS: 00007f8f925fc700(0000) GS:ffff8d148eb80000(0000) knlGS:0000000000000000
[ 128.190998] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 128.190999] CR2: ffffb050839f7000 CR3: 0000800105990000 CR4: 0000000000350ee0
[ 128.191000] Call Trace:
[ 128.191001] <TASK>
[ 128.191004] ? supdrvIOCtl+0x3465/0x38b0 [vboxdrv]
[ 128.191018] VBoxDrvLinuxIOCtl_6_1_32+0x149/0x240 [vboxdrv]
[ 128.191028] __x64_sys_ioctl+0xa8/0xe0
[ 128.191032] do_syscall_64+0x3b/0x80
[ 128.191036] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 128.191038] RIP: 0033:0x7f8fc040dcc7
[ 128.191040] Code: 00 00 00 48 8b 05 c9 91 0c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 99 91 0c 00 f7 d8 64 89 01 48
[ 128.191041] RSP: 002b:00007f8f925fac78 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[ 128.191043] RAX: ffffffffffffffda RBX: 00007f8f7c1e1480 RCX: 00007f8fc040dcc7
[ 128.191044] RDX: 00007f8f7c1e1480 RSI: 0000000000005684 RDI: 0000000000000007
[ 128.191044] RBP: 00007f8f925fac80 R08: 0000000000000000 R09: 00000000fffffffc
[ 128.191045] R10: 0000000000000000 R11: 0000000000000246 R12: 00007f8fb051b7df
[ 128.191046] R13: 00007f8f925fb0e0 R14: 00007f8f925fade0 R15: 0000000000000000
[ 128.191048] </TASK>
[ 128.191049] Modules linked in: ctr(E) ccm(E) rfcomm(E) squashfs(E) zstd_decompress(E) iptable_nat(E) xt_MASQUERADE(E) nf_nat(E) vboxnetadp(OE) vboxnetflt(OE) nf_conntrack(E) vboxdrv(OE) nf_defrag_ipv6(E) cmac(E) nf_defrag_ipv4(E) algif_hash(E) bpfilter(E) algif_skcipher(E) af_alg(E) overlay(E) bnep(E) cpufreq_ondemand(E) 8021q(E) cpufreq_powersave(E) cpufreq_conservative(E) garp(E) stp(E) cpufreq_userspace(E) mrp(E) llc(E) uinput(E) binfmt_misc(E) xfs(E) nls_ascii(E) nls_cp437(E) vfat(E) fat(E) dm_multipath(E) sd_mod(E) sg(E) btusb(E) btrtl(E) btbcm(E) btintel(E) bluetooth(E) jitterentropy_rng(E) sha512_ssse3(E) sha512_generic(E) drbg(E) ansi_cprng(E) ecdh_generic(E) ecc(E) nvidia_drm(POE) iwlmvm(E) drm_kms_helper(E) mac80211(E) edac_mce_amd(E) snd_hda_codec_realtek(E) kvm_amd(E) cec(E) snd_hda_codec_generic(E) syscopyarea(E) ledtrig_audio(E) snd_hda_codec_hdmi(E) sysfillrect(E) sysimgblt(E) kvm(E) snd_hda_intel(E) fb_sys_fops(E) snd_intel_dspcfg(E) libarc4(E) nvidia_modeset(POE)
[ 128.191082] snd_hda_codec(E) iwlwifi(E) snd_hwdep(E) snd_hda_core(E) snd_pcm_oss(E) irqbypass(E) crc32_pclmul(E) snd_mixer_oss(E) ghash_clmulni_intel(E) cfg80211(E) snd_pcm(E) ahci(E) joydev(E) libahci(E) aesni_intel(E) libaes(E) libata(E) dm_mod(E) crypto_simd(E) snd_timer(E) wmi_bmof(E) cryptd(E) snd(E) rapl(E) scsi_mod(E) sp5100_tco(E) pcspkr(E) ccp(E) watchdog(E) soundcore(E) k10temp(E) i2c_piix4(E) scsi_common(E) acpi_cpufreq(E) button(E) wmi(E) nvidia(POE) drm(E) sunrpc(E) tcp_bbr(E) loop(E) fuse(E) configfs(E) ip_tables(E) x_tables(E) autofs4(E) ext4(E) crc16(E) mbcache(E) jbd2(E) raid10(E) raid456(E) libcrc32c(E) crc32c_generic(E) async_raid6_recov(E) async_memcpy(E) async_pq(E) evdev(E) hid_generic(E) usbhid(E) hid(E) raid6_pq(E) async_xor(E) xor(E) async_tx(E) raid1(E) raid0(E) multipath(E) linear(E) md_mod(E) xhci_pci(E) xhci_hcd(E) nvme(E) crc32c_intel(E) usbcore(E) nvme_core(E) t10_pi(E) crc_t10dif(E) crct10dif_generic(E) crct10dif_pclmul(E) crct10dif_common(E)
[ 128.191118] usb_common(E) gpio_amdpt(E) gpio_generic(E)
[ 128.191121] CR2: ffffb050839f7000
[ 128.191122] ---[ end trace e93fcec97286f2fd ]---
[ 128.191124] RIP: 0010:0xffffb050839f7000
[ 128.191125] Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[ 128.191126] RSP: 0018:ffffb0508388fe08 EFLAGS: 00010283
[ 128.191127] RAX: ffffb050839120f0 RBX: ffffb05083a8f010 RCX: ffffb05083891000
[ 128.191128] RDX: ffffb05083911370 RSI: 0000000000000000 RDI: ffff8d0d83852790
[ 128.191129] RBP: ffffb0508388fe98 R08: 0000000000000001 R09: ffffedc080000000
[ 128.191129] R10: ffffffffc306a6e0 R11: 00008003265ae007 R12: 0000000000000024
[ 128.191130] R13: ffffffffc306a6e0 R14: 0000000000000000 R15: ffff8d0d83852790
[ 128.191131] FS: 00007f8f925fc700(0000) GS:ffff8d148eb80000(0000) knlGS:00000
  • 虽然设置了CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=y,还是可以通过设置kernel command line去关闭它的,这里修改如下:
1
2
3
4
5
6
7
8
~$ grep -v '^#\|^$'  /etc/default/grub

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="splash"
GRUB_CMDLINE_LINUX=".... mem_encrypt=off"
GRUB_GFXPAYLOAD_LINUX=keep
  • 关闭AMD SME之后,重启系统再次运行VirtualBox就能正常使用了, 关于AMD SME/SEVQEMU-KVM虚拟机支持的非常好。

Android连接USB时,MTP挂载错误.

  • 有时会出现如:error no such interface 'org.gtk.vfs.mount' on object at path /org/gtk/vfs/mount/1,可以重启系统,也可以尝试如下:
    • GNOME: killall nautilus
    • MATE: killall caja
    • Cinnamon: killall nemo

Samba4 共享问题(SMB/CIFS)

  • Samaba
  • SMB4与前面的版本有一些改动,如:原security = share改成了security = user ; map to guest = Bad User的组合.服务端与客户端的协议版本如果不匹配,也会无法握手连接.可以通过min protocol,client min|max protocol等参数定制.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
grep -v '^#\|^$\|^;\|.*;' /etc/samba/smb.conf
[global]
workgroup = WORKGROUP
netbios name = debian(buster)
# Or set min protocol to NT1, SMB2/3
min protocol = LANMAN2
ea support = yes
interfaces = 127.0.0.0/8 192.168.1.0/24
hosts allow = 127.0.0.1/8 192.168.0.0/16
log file = /var/log/samba/log.%m
log level = 10
max log size = 1000
logging = file
#server role = standalone server
security = user
map to guest = Bad User
[Movies]
comment = Home Directories
path = /home/michael/3TB-DISK/AmuleDir/Incoming/
browseable = yes
available = yes
guest ok = yes
guest only = yes
public = yes
read only = yes

  • 重启,查看服务.

    1
    2
    3
    systemctl restart smbd nmbd
    systemctl status smbd nmbd

  • testparm查验配置文件.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    ~$ testparm
    Registered MSG_REQ_POOL_USAGE
    Registered MSG_REQ_DMALLOC_MARK and LOG_CHANGED
    Load smb config files from /etc/samba/smb.conf
    Processing section "[Movies]"
    Loaded services file OK.
    Server role: ROLE_STANDALONE

    Press enter to see a dump of your service definitions
  • 使用smbclient查看,可以加入-d<number 0-10>调试输出.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
smbclient -m SMB3 -N  -L 192.168.1.100
Unable to initialize messaging context
Anonymous login successful

Sharename Type Comment
--------- ---- -------
Movies Disk Home Directories
IPC$ IPC IPC Service (Samba 4.9.5-Debian)
Reconnecting with SMB1 for workgroup listing.
Anonymous login successful

Server Comment
--------- -------

Workgroup Master
--------- -------
WORKGROUP DEBIAN(BUSTER)

  • 调式smbd,nmbd输出,可以配置smb.conf里面的log level = <0-100>详细级别.
    1
    2
    ~$ smbd  -i -l -S
    ~$ nmbd -i -l -S
  • 错误Error NT_STATUS_IO_TIMEOUT,这个错误最终发现是因服务端系统设置造成的,Samba的配置并在服务端使用smbclient测试都OK,而且在Android手机端是可以查看共享,但是在Win10端与IPad不能连接.最后发现是服务端禁止了ping入造成的.修改成net.ipv4.icmp_echo_ignore_all = 0就可以连接了.

无法进虚拟终端的问题(Ctrl-Alt-f[1-6])

  • NVIDIA
  • NVIDIA/Tips and tricks
  • Kernel mode setting
  • The Framebuffer Console
  • 有时新安装了新的(Nvidia)显卡驱动后,发现无法切换到虚拟的终端里,只能使用Ctrl-Alt-F7的终端.确认一下/etc/default/grub内有如内容:
    1
    2
    3
    4
    5
    6
    7
    grep -v '^#\|^$' /etc/default/grub
    GRUB_DEFAULT=0
    GRUB_TIMEOUT=5
    GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
    GRUB_CMDLINE_LINUX_DEFAULT="splash"
    GRUB_CMDLINE_LINUX="resume=UUID=ae170b9d-832a-46a7-beca-5fb7c9329b1b nomodeset"
    GRUB_GFXPAYLOAD_LINUX=keep
  • 加入nomodeset,开启GRUB_GFXPAYLOAD_LINUX=keep后运行update-grub,重启系统可能会解决此问题.

一起无法重启networking服务的问题

1
2
3
~$ sudo systemctl restart networking.service
Job for networking.service failed because the control process exited with error code.
See "systemctl status networking.service" and "journalctl -xe" for details.
  • status
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
~$ sudo systemctl status networking.service
● networking.service - Raise network interfaces
Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2021-04-14 22:42:17 CST; 48s ago
Docs: man:interfaces(5)
Process: 10535 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=1/FAILURE)
Main PID: 10535 (code=exited, status=1/FAILURE)

Apr 14 22:42:17 debian ifup[10535]: run-parts: executing /etc/network/if-up.d/avahi-autoipd
Apr 14 22:42:17 debian ifup[10535]: run-parts: executing /etc/network/if-up.d/dhcpcd
Apr 14 22:42:17 debian ifup[10535]: run-parts: executing /etc/network/if-up.d/ethtool
Apr 14 22:42:17 debian ifup[10535]: run-parts: executing /etc/network/if-up.d/ip
Apr 14 22:42:17 debian ifup[10535]: run-parts: executing /etc/network/if-up.d/mountnfs
Apr 14 22:42:17 debian ifup[10535]: run-parts: executing /etc/network/if-up.d/uml-utilities
Apr 14 22:42:17 debian ifup[10535]: run-parts: executing /etc/network/if-up.d/wpasupplicant
Apr 14 22:42:17 debian systemd[1]: networking.service: Main process exited, code=exited, status=1/FAILURE
Apr 14 22:42:17 debian systemd[1]: networking.service: Failed with result 'exit-code'.
Apr 14 22:42:17 debian systemd[1]: Failed to start Raise network interfaces.

  • journatctl

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    sudo journalctl -u networking.service
    [...]
    Apr 14 19:26:20 debian ifup[916]: dnsmasq: failed to create listening socket for port 53: Address already in use
    Apr 14 19:26:20 debian dnsmasq[7338]: failed to create listening socket for port 53: Address already in use
    Apr 14 19:26:20 debian dnsmasq[7338]: FAILED to start up
    Apr 14 19:26:20 debian ifup[916]: ifup: failed to bring up wlan1
    [...]

    Apr 14 19:26:20 debian ifup[916]: run-parts: executing /etc/network/if-pre-up.d/wpasupplicant
    Apr 14 19:26:20 debian ifup[916]: /sbin/ip addr add 192.168.3.1/255.255.255.0 broadcast 192.168.3.255 dev wlan2 label wlan2
    Apr 14 19:26:20 debian ifup[916]: Cannot find device "wlan2"
    Apr 14 19:26:20 debian ifup[916]: ifup: failed to bring up wlan2
    Apr 14 19:26:20 debian ifup[916]: ifup: configuring interface br-lan=br-lan (inet)
    Apr 14 19:26:20 debian ifup[916]: ifup: missing required variable: address
    Apr 14 19:26:20 debian ifup[916]: ifup: missing required configuration variables for interface br-lan/inet
    Apr 14 19:26:20 debian ifup[916]: ifup: failed to bring up br-lan
    Apr 14 19:26:20 debian ifup[916]: ifup: configuring interface wlan0=wlan0 (inet)

  • search 53 service

    1
    2
    3
    4
    5
    ~$ sudo netstat -unlp | grep "53"
    udp 0 0 224.0.0.251:5353 0.0.0.0:* 27555/brave --type=
    udp 0 0 224.0.0.251:5353 0.0.0.0:* 27555/brave --type=
    udp 0 0 127.0.0.1:53 0.0.0.0:* 1/init
    udp6 0 0 ::1:53 :::* 1/init
1
2
3
4
5
6
7
8
 systemctl list-sockets | grep "53"
127.0.0.1:53 kresd.socket kresd@1.service
127.0.0.1:53 kresd.socket kresd@1.service
127.0.0.1:853 kresd-tls.socket kresd@1.service
[::1]:53 kresd.socket kresd@1.service
[::1]:53 kresd.socket kresd@1.service
[::1]:853 kresd-tls.socket kresd@1.service

  • 查看系统启动时的所有错误
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    ~$ systemctl --no-pager --all --state=failed
    UNIT LOAD ACTIVE SUB DESCRIPTION
    ● dokku-retire.service loaded failed failed Dokku retire service
    ● networking.service loaded failed failed Raise network interfaces
    ● rc-local.service loaded failed failed /etc/rc.local Compatibility
    ● systemd-modules-load.service loaded failed failed Load Kernel Modules

    LOAD = Reflects whether the unit definition was properly loaded.
    ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
    SUB = The low-level unit activation state, values depend on unit type.

    4 loaded units listed.
    To show all installed unit files use 'systemctl list-unit-files'.

让远端的GUI运行到本机界面

SSH X Forwarding

  • 这个比较简单,在/etc/ssh/sshd_config要有如下设置,再用要ssh -X user@<remote_host>登录,运行一个类X的程序,就会出现在本机.
1
2
3
AllowTcpForwarding yes
X11Forwarding yes
X11UseLocalhost yes
  • 一般以查看<remote_host>里的echo $DISPLAY变量是localhost:10.0.

X Server tcp

  • 这里以lightdm为例,其它:GDM,XDM类似.确保/etc/lightdm/lightdm.conf里有以下行

    1
    2
    3
    4
    5
    6
    7
    8
    ~$ grep -v '^#\|^$' /etc/lightdm/lightdm.conf
    [LightDM]
    [Seat:*]
    xserver-allow-tcp=true
    [XDMCPServer]
    enabled=true
    port=177
    [VNCServer]
  • 重启本机的lightdm服务.就会有发现Xservertcp listen. 一般来说使用SSH X Fowarding就可以满足很多场景需求.下面这个例就必须使用Xserver tcp,如果使用SSH X Forwarding就出现gtk initialization failed.

    1
    docker run  --device /dev/kvm    -e RAM=6 -p 50922:10022 -v $XAUTH:/root/.Xauthority -e DISPLAY=<remote_host>:0.0 docker-osx:latest
  • 上面两个方式还是提示无权连接时,在本机运行xhost +再测试.

Linux下IPv6路由丢失

  • Debian下面遇到过种,不知道过了多久,系统内wlan0网卡的IPv6路由不见,但是IPv6的地址还在.后来发现是sysctl配置的问题,配置成下面就解决了:
    1
    2
    net.ipv6.conf.default.autoconf = 0
    net.ipv6.conf.wlan0.autoconf = 0

Knot DNS缓存问题

  • 之前使用dnsmasqdhcp+dns同时还支持IPv6,不知何时dnsmasq启动时,提示53端口被占,检查发现还是被1/init进程占住.这里无法通过/proc/1/cmdline查找.通过find /usr/lib/systemd/system -exec grep -nH "127.0.0.1:53" {} \;发现在kresd.socket文件内有如下设置.
1
2
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      1/init
tcp 0 0 127.0.0.1:853 0.0.0.0:* LISTEN 1/init

Debian 10使用systemed运行/etc/rc.local开机自启命令

  • 创建systemed服务描述文件.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    ~$ cat>/etc/systemd/system/rc-local.service<<EOF
    [Unit]
    Description=/etc/rc.local Compatibility
    Documentation=man:systemd-rc-local-generator(8)
    ConditionFileIsExecutable=/etc/rc.local
    ConditionPathExists=/etc/rc.local
    [Service]
    Type=forking
    ExecStart=/etc/rc.local start
    TimeoutSec=0
    RemainAfterExit=yes
    StandardOutput=tty
    SysVStartPriority=99
    [Install]
    WantedBy=multi-user.target
    EOF
  • 创建运行脚本文件,可以把需要运行的命令加入到exit 0之前.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    ~$cat>/etc/rc.local<<EOF
    #!/bin/sh -e
    #
    # rc.local
    #
    # This script is executed at the end of each multiuser runlevel.
    # Make sure that the script will "exit 0" on success or any other
    # value on error.
    #
    # In order to enable or disable this script just change the execution
    # bits.
    #
    # By default this script does nothing.

    exit 0
    EOF
  • 分配运行权限,与开机自运行命令

    1
    2
    3
    4
    ~$ chmod +x /etc/rc.local
    ~$ systemctl enable rc-local
    ~$ systemctl start rc-local

Linux下安装Xrdp服务

  • Ubuntu 18.04: Connect to MATE desktop environment via XRDP
    1
    ~$ apt-get install xrdp
  • xRDP uses the /etc/ssl/private/ssl-cert-snakeoil.key
    file which belongs to the ssl-cert group. As such we need to add the xRDP user to that group:
    1
    ~$ sudo adduser xrdp ssl-cert
  • 如果出现Could not acquire name on session bus. 需要如下设置:
    1
    2
    3
    4
    cat <<EOF> ~/.xsession
    unset DBUS_SESSION_BUS_ADDRESS
    mate-session
    EOF

Linux下运行x11vnc

  • X11vnc

  • x11vnc可以支持共享与控制的功能.简单运行如下

    1
    ssh -t -L 5900:localhost:5900 remote_vnchost 'VNC_PASSWORD=123456 x11vnc -localhost -display :0'
  • 如遇到下面的错误.

    1
    2
    3
    4
    5
    27/09/2021 20:47:05 Xinerama: no blackouts needed (screen fills rectangle)
    27/09/2021 20:47:05
    27/09/2021 20:47:05 shmget(fullscreen) failed.
    27/09/2021 20:47:05 shmget: No space left on device

  • 查看内核设置

    1
    2
    3
    4
    5
    6
    7
    sudo sysctl -a | grep "shm"
    kernel.shm_next_id = -1
    kernel.shm_rmid_forced = 0
    kernel.shmall = 18446744073692774399
    kernel.shmmax = 18446744073692774399
    kernel.shmmni = 4096
    vm.hugetlb_shm_group = 0
  • 需要把kernel.shmmni增加到8192.

systemd

1
2
3
4
5
6
7
8
~$ cat /etc/systemd/system/x11vnc.service
[Service]
ExecStart=
ExecStart=/usr/bin/x11vnc -many -display :0 -auth /var/run/lightdm/root/:0

[Install]
WantedBy=graphical.target

rime同文输入法

  • Rime是一个跨平台的输入框架.它的核心(或者说’后端’)是librime引擎,不同平台的前端实现共用这个后端.目前在GitHub RIME组织中维护的前端实现有三个:
    用于 Windows 的 Weasel 「小狼毫」;
    用于 macOS 的 Squirrel 「鼠须管」;
    用于 Linux 的 ibus-rime 「中州韵」——Rime 本身的中文名字也叫「中州韵输入法引擎」.

  • 同步之后可以看到文件夹下的相关配置,基本上和桌面版的RIME是一致的:
    default.yaml,各输入方案共享的全局配置
    default.custom.yaml,(可选)对 default.yaml 的修改,不会随着客户端的更新而被覆盖,所以对default的修改可以通过patch的方式放入该配置文件
    xxx.schema.yaml,xxx 输入方案的配置
    xxx.custom.yaml,(可选)对 xxx.schema.yaml 的自定义修改
    xxx.dict.yaml, xxx 输入方案的词库(字典)

  • 这里主要是记录网上链接,各种配置与词库都是热心网友供献,你可以稍作修改,或者不作修改都能有很好的体验.如果自已有空也可以定制出自已的风格,且可以同步各种平台上使用.

  • Android 上的 RIME 输入法 trime 同文输入法使用

  • 使用 Trime 在 Android 上输入五笔

  • 同文安卓輸入法平臺/Trime: Rime IME for Android

  • RIME | 中州韻輸入法引擎

  • Rime 定製指南

  • Rime 输入法指北

Youtube-dl使用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
#!/bin/bash

# youtube-dl -i --playlist-reverse -j --flat-playlist 'https://www.youtube.com/playlist?list=$1' | jq -r '.id + " # " + .title' | sed 's_^_https://youtube.com/v/_'
TMP_IFS=$IFS
IFS=$(echo -en "\n\b")
export UA='Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/0123456789 Safari/537.36'
export https_proxy=http://127.0.0.1:8123
export http_proxy=http://127.0.0.1:8123
export DL=/usr/local/bin/youtube-dl
export SUBLANG="en"
alias ylp="/usr/local/bin/youtube-dl --user-agent '${UA}'"


function get_info()
{
${DL} --user-agent "${UA}" -F $1
}

function download_file()
{
echo "start download file from URL $1 with sub lang $SUBLANG"
URL=$(echo "$1" | awk -F'#' '{print $1}')
TITLE=$(echo "$1" | awk -F'#' '{print $2}')".mp4"
INFO=$(get_info ${URL})
AUDIO=$(printf "${INFO}" | grep "audio only"| awk 'END{print $1}')
VIDEO=$(printf "${INFO}" | grep "x1080" | awk 'END{print $1}')
AV=${VIDEO}+${AUDIO}
${DL} --user-agent "${UA}" -f $AV --sub-lang ${SUBLANG} --write-auto-sub --convert-subs=srt --external-downloader aria2c --external-downloader-args '-c -j 16 -x 16 -s 16 -k 10M' "$URL"
}

function download_list()
{
GET_PLAYLIST=$(${DL} -i --playlist-reverse -j --flat-playlist "https://www.youtube.com/playlist?list=$1" | jq -r '.id + "#" + .title')
echo "${GET_PLAYLIST}"
for line in ${GET_PLAYLIST};
do
download_file $line
done
}

if [[ -z $2 ]]; then
SUBLANG=$2
fi
case "${#1}" in
11) download_file $1;;
34) download_list $1;;
*)
echo "got wrong url length, exit.....";;
esac
IFS=$TMP_IFS

通过ADB安装程序到小米电视机

  • 一般旧的电视视MiTv通过设置设置-->账号与安全-->允许安装未知来源的应用后,再把相应的apk复现到U盘接入后,可以直选择到文件,进行安装。还有另一种方式,稍微麻烦一的。

  • 开启debug调式模式:设置-->关于-->产品型号遥控连续按5下OK键.打开后,界面有提示。再通过设置设置-->账号与安全-->允许ADB调试允许调试。下面就是adb的操作了。如:本电视机的IP是192.168.1.130

1
2
3
4
5
6
~$ adb connect 192.168.1.130:5555

$ adb devices
List of devices attached
192.168.1.130:5555 device

  • adb shell

  • 上传程序

1
~$ adb push kodi-20.1-Nexus-armeabi-v7a.apk /mnt/sdcard/Download
  • 安装程序
1
~$ adb install kodi-20.1-Nexus-armeabi-v7a.apk
  • 出现错误Failure [INSTALL_FAILED_OLDER_SDK],说明系统版本太旧,SDK不能兼容,旧的电视机只能安装降低版本的apk

HP笔记本TouchPad问题

  • Dell Precision 7550 physical mouse buttons behaving like clickpad

  • Clickpad left click not working on Ubuntu 20.04 when touchpad is not touched

  • 新装有笔记本的电脑的TouchPad不能轻敲代替点击,需要按下物理键才行。

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    ~$ xinput list
    \u23a1 Virtual core pointer id=2 [master pointer (3)]
    \u239c \u21b3 Virtual core XTEST pointer id=4 [slave pointer (2)]
    \u239c \u21b3 SYNA30AA:00 06CB:CDEB Mouse id=12 [slave pointer (2)]
    \u239c \u21b3 SYNA30AA:00 06CB:CDEB Touchpad id=13 [slave pointer (2)]
    \u23a3 Virtual core keyboard id=3 [master keyboard (2)]
    \u21b3 Virtual core XTEST keyboard id=5 [slave keyboard (3)]
    \u21b3 Power Button id=6 [slave keyboard (3)]
    \u21b3 Video Bus id=7 [slave keyboard (3)]
    \u21b3 Video Bus id=8 [slave keyboard (3)]
    \u21b3 Power Button id=9 [slave keyboard (3)]
    \u21b3 Sleep Button id=10 [slave keyboard (3)]
    \u21b3 HP HD Camera: HP HD Camera id=11 [slave keyboard (3)]
    \u21b3 AT Translated Set 2 keyboard id=14 [slave keyboard (3)]
1
2
3
4
~$ xinput list-props "SYNA30AA:00 06CB:CDEB Touchpad" | grep -i click
Synaptics ClickPad (352): 1
Synaptics Click Action (364): 1, 0, 0

  • Synaptics ClickPad设置为0.就能正常使用了。
    1
    102  xinput set-prop "SYNA30AA:00 06CB:CDEB Touchpad" "Synaptics ClickPad" 0

Vim无法使用YouCompleteMe的插件

  • 已经进入到目录里并且安装:~/.vim/bundle/YouCompleteMe && ./install.py --all,但是运行还是出现如下的错误。
1
2
3
4
~$ vim
YouCompleteMe unavailable: requires Vim compiled with Python (3.8.0+) support.
Press ENTER or type command to continue

  • 进入vim,运行::echo has("python_dynamic"),:echo has("python_dynamic")都是返回0,说明没有加载到python,可以运行::help python-dynamic查看相关的帮助文档。
  • 运行vim --version查看详情
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
~$ vim --version
VIM - Vi IMproved 9.0 (2022 Jun 28, compiled May 04 2023 10:24:44)
Included patches: 1-1378, 1499
Modified by team+vim@tracker.debian.org
Compiled by team+vim@tracker.debian.org
Huge version without GUI. Features included (+) or not (-):
+acl +file_in_path +mouse_urxvt -tag_any_white
+arabic +find_in_path +mouse_xterm -tcl
+autocmd +float +multi_byte +termguicolors
+autochdir +folding +multi_lang +terminal
-autoservername -footer -mzscheme +terminfo
-balloon_eval +fork() +netbeans_intg +termresponse
+balloon_eval_term +gettext +num64 +textobjects
-browse -hangul_input +packages +textprop
++builtin_terms +iconv +path_extra +timers
+byte_offset +insert_expand -perl +title
+channel +ipv6 +persistent_undo -toolbar
+cindent +job +popupwin +user_commands
-clientserver +jumplist +postscript +vartabs
-clipboard +keymap +printer +vertsplit
+cmdline_compl +lambda +profile +vim9script
+cmdline_hist +langmap -python +viminfo
+cmdline_info +libcall -python3 +virtualedit
+comments +linebreak +quickfix +visual
+conceal +lispindent +reltime +visualextra
+cryptv +listcmds +rightleft +vreplace
+cscope +localmap -ruby +wildignore
+cursorbind -lua +scrollbind +wildmenu
+cursorshape +menu +signs +windows
+dialog_con +mksession +smartindent +writebackup
+diff +modify_fname +sodium -X11
+digraphs +mouse -sound -xfontset
-dnd -mouseshape +spell -xim
-ebcdic +mouse_dec +startuptime -xpm
+emacs_tags +mouse_gpm +statusline -xsmp
+eval -mouse_jsbterm -sun_workshop -xterm_clipboard
+ex_extra +mouse_netterm +syntax -xterm_save
+extra_search +mouse_sgr +tag_binary
-farsi -mouse_sysmouse -tag_old_static
system vimrc file: "/etc/vim/vimrc"
user vimrc file: "$HOME/.vimrc"
2nd user vimrc file: "~/.vim/vimrc"
user exrc file: "$HOME/.exrc"
defaults file: "$VIMRUNTIME/defaults.vim"
fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -Wdate-time -g -O2 -ffile-prefix-map=/build/vim-JA6Vy9/vim-9.0.1378=. -fstack-protector-strong -Wformat -Werror=format-security -DSYS_VIMRC_FILE=\"/etc/vim/vimrc\" -DSYS_GVIMRC_FILE=\"/etc/vim/gvimrc\" -D_REENTRANT -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: gcc -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim -lm -ltinfo -lselinux -lsodium -lacl -lattr -lgpm

  • 上面显示-python,-python3. 安装apt-get install vim-nox vim-youcompleteme就能正常使用了。

谢谢支持

GCC编译相关

GCC 编译出错

1
2
3
4
5
6
7
8
In file included from /home/michael/3TB-DISK/Embedded-System/NuttX/buildroot/toolchain_build_arm_nofpu/gcc-4.8.5/gcc/cp/except.c:1008:0:
cfns.gperf: In function ‘const char* libc_name_p(const char*, unsigned int)’:
cfns.gperf:101:1: error: ‘const char* libc_name_p(const char*, unsigned int)’ redeclared inline with ‘gnu_inline’ attribute
cfns.gperf:26:14: note: ‘const char* libc_name_p(const char*, unsigned int)’ previously declared here
cfns.gperf: At global scope:
cfns.gperf:26:14: warning: inline function ‘const char* libc_name_p(const char*, unsigned int)’ used but never defined
Makefile:1059: recipe for target 'cp/except.o' failed
make[3]: *** [cp/except.o] Error 1
  • 处理方法
  • 添加条件宏定义到这两个文件中**gcc-4.8.5/gcc/cp/cfns.{gperf,h} **
1
2
3
4
5
6
7
8
9
10
11
12
19	#ifdef __GNUC__
20 __inline
21 #endif
22 static unsigned int hash (const char *, unsigned int);
23 #ifdef __GNUC__
24 __inline
25 #ifdef __GNUC_STDC_INLINE__
26 __attribute__ ((__gnu_inline__))
27 #endif
28 #endif
29 const char * libc_name_p (const char *, unsigned int);

编译 gcc texi 出错如下:

1
2
3
4
5
6
7
8
/home/michael/3TB-DISK/Embedded-System/NuttX/buildroot/toolchain_build_arm_nofpu/gcc-4.8.5/gcc/doc/gcc.texi:88: warning: @tex should only appear at the beginning of a line
/home/michael/3TB-DISK/Embedded-System/NuttX/buildroot/toolchain_build_arm_nofpu/gcc-4.8.5/gcc/doc/gcc.texi:208: no matching `@end tex'
/home/michael/3TB-DISK/Embedded-System/NuttX/buildroot/toolchain_build_arm_nofpu/gcc-4.8.5/gcc/doc/gcc.texi:208: no matching `@end multitable'
/home/michael/3TB-DISK/Embedded-System/NuttX/buildroot/toolchain_build_arm_nofpu/gcc-4.8.5/gcc/doc/gcc.texi:208: no matching `@end titlepage'
Makefile:4353: recipe for target 'doc/gcc.info' failed
make[3]: *** [doc/gcc.info] Error 1
make[3]: Leaving directory '/home/michael/3TB-DISK/Embedded-System/NuttX/buildroot/toolchain_build_arm_nofpu/gcc-4.8.5-build/gcc'
Makefile:3889: recipe for target 'all-gcc' failed
  • 处理方法
1
2
3
$cd /home/michael/3TB-DISK/Embedded-System/NuttX/buildroot/toolchain_build_arm_nofpu/gcc-4.8.5-build/gcc
$ vi Makefile
注释 doc: 这一行,跳过编译doc相关的东西

C++ 函数定义错误

1
2
3
4
5
libxx_new.cxx:66:40: error: 'operator new' takes type 'size_t' ('unsigned int') as first parameter [-fpermissive]
void *operator new(unsigned long nbytes)
^
Makefile:111: recipe for target 'libxx_new.o' failed
make[1]: *** [libxx_new.o] Error 1
  • 处理方法
  • Linux 下要定义成
1
void *operator new(size_t nbytes)
  • MinGW 下要定义成
1
2
3
4
5
#ifdef CONFIG_CXX_NEWLONG
void *operator new(unsigned long nbytes)
#else
void *operator new(unsigned int nbytes)
#endif

Intel Corporation Comet Lake PCH-LP cAVS集成声卡问题

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
~$ lspci -v -s 00:1f.3
00:1f.3 Multimedia audio controller: Intel Corporation Comet Lake PCH-LP cAVS
Subsystem: Hewlett-Packard Company Comet Lake PCH-LP cAVS
Flags: bus master, fast devsel, latency 64, IRQ 151
Memory at 4000108000 (64-bit, non-prefetchable) [size=16K]
Memory at 4000000000 (64-bit, non-prefetchable) [size=1M]
Capabilities: <access denied>
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel, snd_sof_pci_intel_cnl

~$ pactl list cards
Card #0
Name: alsa_card.pci-0000_00_1f.3
Driver: module-alsa-card.c
Owner Module: 6
Properties:
alsa.card = "0"
alsa.card_name = "HDA Intel PCH"
alsa.long_card_name = "HDA Intel PCH at 0x4000108000 irq 151"
alsa.driver_name = "snd_hda_intel"
device.bus_path = "pci-0000:00:1f.3"
sysfs.path = "/devices/pci0000:00/0000:00:1f.3/sound/card0"
device.bus = "pci"
device.vendor.id = "8086"
device.vendor.name = "Intel Corporation"
device.product.id = "02c8"
device.product.name = "Comet Lake PCH-LP cAVS"
device.form_factor = "internal"
device.string = "0"
device.description = "Built-in Audio"
module-udev-detect.discovered = "1"
device.icon_name = "audio-card-pci"
Profiles:
[...]

1
2
3
4
5
6
7
8
9
10
~$ pacmd list-sources | grep -e index -e name: -e muted -e device.description
index: 0
name: <alsa_output.pci-0000_00_1f.3.analog-stereo.monitor>
muted: no
device.description = "Monitor of Built-in Audio Analog Stereo"
* index: 1
name: <alsa_input.pci-0000_00_1f.3.analog-stereo>
muted: no
device.description = "Built-in Audio Analog Stereo"

1
2
3
4
~$ amixer scontrols
Simple mixer control 'Master',0
Simple mixer control 'Capture',0

  • aplay -l
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
~$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 1: PCH [HDA Intel PCH], device 0: ALC236 Analog [ALC236 Analog]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: PCH [HDA Intel PCH], device 3: HDMI 0 [HDMI 0]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: PCH [HDA Intel PCH], device 7: HDMI 1 [HDMI 1]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: PCH [HDA Intel PCH], device 8: HDMI 2 [HDMI 2]
Subdevices: 1/1
Subdevice #0: subdevice #0

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
sudo journalctl -k | grep -Ei "ALSA|HDA|sof[-]|HDMI|snd[_-]|sound|hda.codec|hda.intel"
[sudo] password for jh:
10月 26 00:02:20 jh-hpzhan66pro14g3 kernel: pci 0000:01:00.0: Enabling HDA controller
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: snd_hda_intel 0000:00:1f.3: enabling device (0000 -> 0002)
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: snd_hda_intel 0000:00:1f.3: bound 0000:00:02.0 (ops i915_audio_component_bind_ops [i915])
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC236: line_outs=1 (0x14/0x0/0x0/0x0/0x0) type:speaker
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x21/0x0/0x0/0x0/0x0)
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: snd_hda_codec_realtek hdaudioC1D0: inputs:
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: snd_hda_codec_realtek hdaudioC1D0: Internal Mic=0x19
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: snd_hda_codec_realtek hdaudioC1D0: Mic=0x18
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: input: HDA Intel PCH Mic as /devices/pci0000:00/0000:00:1f.3/sound/card1/input16
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: input: HDA Intel PCH Headphone as /devices/pci0000:00/0000:00:1f.3/sound/card1/input17
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: input: HDA Intel PCH HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:1f.3/sound/card1/input18
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: input: HDA Intel PCH HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:1f.3/sound/card1/input19
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: input: HDA Intel PCH HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:1f.3/sound/card1/input20
10月 26 00:02:21 jh-hpzhan66pro14g3 kernel: alsactl[624]: memfd_create() called without MFD_EXEC or MFD_NOEXEC_SEAL set

1
2
3
4
5
6
7
8
9
10
11
12
13
14
~$ dmesg  | grep 'snd\|firmware\|audio'
[ 5.271850] i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/kbl_dmc_ver1_04.bin (v1.4)
[ 8.753092] snd_hda_intel 0000:00:1f.3: enabling device (0000 -> 0002)
[ 8.753414] snd_hda_intel 0000:00:1f.3: bound 0000:00:02.0 (ops i915_audio_component_bind_ops [i915])
[ 8.768179] iwlwifi 0000:00:14.3: loaded firmware version 77.bd067429.0 QuZ-a0-hr-b0-77.ucode op_mode iwlmvm
[ 8.907449] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC236: line_outs=1 (0x14/0x0/0x0/0x0/0x0) type:speaker
[ 8.907455] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 8.907458] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x21/0x0/0x0/0x0/0x0)
[ 8.907460] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
[ 8.907461] snd_hda_codec_realtek hdaudioC1D0: inputs:
[ 8.907463] snd_hda_codec_realtek hdaudioC1D0: Internal Mic=0x19
[ 8.907465] snd_hda_codec_realtek hdaudioC1D0: Mic=0x18
[ 9.313962] Bluetooth: hci0: Found device firmware: intel/ibt-19-0-4.sfi

1
~$ sudo apt-get install firmware-sof-signed  firmware-realtek -y
1
2
3
4
5
6
~$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 1: PCH [HDA Intel PCH], device 0: ALC236 Analog [ALC236 Analog]
Subdevices: 0/1
Subdevice #0: subdevice #0


谢谢支持

  • 本人有空会折腾编译最新的Linux内核,使用的笔记本电脑又AMD的平台,AMD显卡fglrx驱动一直是一个巨大的坑,没有一次可以完全正常编译的,对于不同的内核本版要打无数的补丁才能编译成功,且它的驱动不支持VAPDU特性.对于不是很新的显卡用开源的radeon驱动都能很好的使用.

  • [参考链接]

  • Mesa-12.0.1

安装依赖库

LLVM-3.8

下载LLVM3.8后解压到
/usr/local/clang+llvm-3.8.0-x86_64-linux-gnu-debian8

  • libdrm-2.4.68

  • 下载后解压编译如下:

    1
    2
    3
    $ ../configure --enable-udev --enable-radeon --enable-amdgpu --disable-intel --disable-nouveau --disable-vmwgfx
    [...]
    $ make && sudo make install
  • 下面这些依赖的库文件很简单的编译,直接解压进去,./configure && make && make install 就可以了,

  • 安装的目录在/usr/local,如果编译过程碰到错误,就是少某些头文件,安装它就可以继续编译了.

安装libva

1
2
$  ../configure --enable-x11 --enable-drm  --enable-egl --enable-glx
$ make && make install

安装MESA

mesa-12.0.1

1
2
3
4
5
6
7
8
9
10
11
$ tar xvf mesa-12.0.1.tar.gz
$ cd mesa-12.0.1 && mkdir build && cd build
$ GLL_DRV="r300,r600,radeonsi,swrast"
$ ../autogen.sh --enable-dri --enable-glx --enable-vdpau --enable-opencl --enable-opencl-icd \
--enable-glx-tls --with-llvm-prefix=/usr/local/clang+llvm-3.8.0-x86_64-linux-gnu-debian8 \
--disable-llvm-shared-libs --enable-egl --enable-gbm \
--enable-texture-float --enable-xa --with-gallium-drivers=$GLL_DRV \
--with-dri-drivers=radeon --with-egl-platforms=drm,x11 --enable-gles1 \
--enable-gles2 --enable-osmesa --enable-va --with-osmesa-bits=32 --enable-gallium-extra-hud
[....]
$ make install
  • 替换系统的MESA库
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    $cd /usr/lib/x86_64-linux-gnu/dri
    debian:/usr/lib/x86_64-linux-gnu/dri$ ls -l
    total 82908
    -rw-r--r-- 1 root root 22504 Oct 24 2014 dummy_drv_video.so
    -rw-r--r-- 5 root root 5505624 Aug 20 2015 i915_dri.so
    -rw-r--r-- 5 root root 5505624 Aug 20 2015 i965_dri.so
    -rw-r--r-- 7 root root 8175344 Aug 20 2015 kms_swrast_dri.so
    -rw-r--r-- 7 root root 8175344 Aug 20 2015 nouveau_dri.so
    lrwxrwxrwx 1 root root 18 Aug 8 2014 nouveau_drv_video.so -> vdpau_drv_video.so
    -rw-r--r-- 5 root root 5505624 Aug 20 2015 nouveau_vieux_dri.so
    lrwxrwxrwx 1 root root 18 Aug 8 2014 nvidia_drv_video.so -> vdpau_drv_video.so
    -rw-r--r-- 5 root root 5505624 Aug 20 2015 r200_dri.so
    -rw-r--r-- 7 root root 8175344 Aug 20 2015 r300_dri.so
    lrwxrwxrwx 1 root root 30 Aug 6 23:20 r600_dri.so -> /usr/local/lib/dri/r600_dri.so
    -rw-r--r-- 7 root root 8175344 Aug 20 2015 r600_dri.so.org
    lrwxrwxrwx 1 root root 18 Aug 8 2014 r600_drv_video.so -> vdpau_drv_video.so
    -rw-r--r-- 5 root root 5505624 Aug 20 2015 radeon_dri.so
    -rw-r--r-- 7 root root 8175344 Aug 20 2015 radeonsi_dri.so
    lrwxrwxrwx 1 root root 18 Aug 8 2014 radeonsi_drv_video.so -> vdpau_drv_video.so
    lrwxrwxrwx 1 root root 18 Aug 8 2014 s3g_drv_video.so -> vdpau_drv_video.so
    lrwxrwxrwx 1 root root 32 Aug 7 00:46 swrast_dri.so -> /usr/local/lib/dri/swrast_dri.so
    -rw-r--r-- 7 root root 8175344 Aug 20 2015 swrast_dri.so.org
    -rw-r--r-- 1 root root 97056 Aug 8 2014 vdpau_drv_video.so
    -rw-r--r-- 7 root root 8175344 Aug 20 2015 vmwgfx_dri.so
1
2
3
4
5
6
7
8
9
10
11
12
13
14
ls /usr/lib/x86_64-linux-gnu/libdrm_* -l
lrwxrwxrwx 1 root root 37 Dec 11 19:14 /usr/lib/x86_64-linux-gnu/libdrm_amdgpu.so -> /usr/local/lib/libdrm_amdgpu.so.1.0.0
-rw-r--r-- 1 root root 206760 Oct 6 2014 /usr/lib/x86_64-linux-gnu/libdrm_intel.a
lrwxrwxrwx 1 root root 30 Dec 11 13:30 /usr/lib/x86_64-linux-gnu/libdrm_intel.so -> /usr/local/lib/libdrm_intel.so
lrwxrwxrwx 1 root root 21 Oct 6 2014 /usr/lib/x86_64-linux-gnu/libdrm_intel.so.1 -> libdrm_intel.so.1.0.0
-rw-r--r-- 1 root root 139328 Oct 6 2014 /usr/lib/x86_64-linux-gnu/libdrm_intel.so.1.0.0
-rw-r--r-- 1 root root 31552 Oct 6 2014 /usr/lib/x86_64-linux-gnu/libdrm_nouveau.a
lrwxrwxrwx 1 root root 32 Dec 11 13:30 /usr/lib/x86_64-linux-gnu/libdrm_nouveau.so -> /usr/local/lib/libdrm_nouveau.so
lrwxrwxrwx 1 root root 23 Oct 6 2014 /usr/lib/x86_64-linux-gnu/libdrm_nouveau.so.2 -> libdrm_nouveau.so.2.0.0
-rw-r--r-- 1 root root 26896 Oct 6 2014 /usr/lib/x86_64-linux-gnu/libdrm_nouveau.so.2.0.0
-rw-r--r-- 1 root root 71708 Oct 6 2014 /usr/lib/x86_64-linux-gnu/libdrm_radeon.a
lrwxrwxrwx 1 root root 37 Dec 11 19:13 /usr/lib/x86_64-linux-gnu/libdrm_radeon.so -> /usr/local/lib/libdrm_radeon.so.1.0.1
lrwxrwxrwx 1 root root 37 Dec 11 19:13 /usr/lib/x86_64-linux-gnu/libdrm_radeon.so.1 -> /usr/local/lib/libdrm_radeon.so.1.0.1
-rw-r--r-- 1 root root 51688 Oct 6 2014 /usr/lib/x86_64-linux-gnu/libdrm_radeon.so.1.0.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
for i in {drm,egl,glx,va,tpi,wayland,x11};do sudo ln -svf /usr/local/lib/libva-$i.so.1.3904.0 libva-$i.so.1  ;done
‘libva-drm.so.1’ -> ‘/usr/local/lib/libva-drm.so.1.3904.0’
‘libva-egl.so.1’ -> ‘/usr/local/lib/libva-egl.so.1.3904.0’
‘libva-glx.so.1’ -> ‘/usr/local/lib/libva-glx.so.1.3904.0’
‘libva-va.so.1’ -> ‘/usr/local/lib/libva-va.so.1.3904.0’
‘libva-tpi.so.1’ -> ‘/usr/local/lib/libva-tpi.so.1.3904.0’
‘libva-wayland.so.1’ -> ‘/usr/local/lib/libva-wayland.so.1.3904.0’
‘libva-x11.so.1’ -> ‘/usr/local/lib/libva-x11.so.1.3904.0’
michael@debian:/usr/lib/x86_64-linux-gnu$ for i in {drm,egl,glx,va,tpi,wayland,x11};do sudo ln -svf /usr/local/lib/libva-$i.so.1.3904.0 libva-$i.so ;done
‘libva-drm.so’ -> ‘/usr/local/lib/libva-drm.so.1.3904.0’
‘libva-egl.so’ -> ‘/usr/local/lib/libva-egl.so.1.3904.0’
‘libva-glx.so’ -> ‘/usr/local/lib/libva-glx.so.1.3904.0’
‘libva-va.so’ -> ‘/usr/local/lib/libva-va.so.1.3904.0’
‘libva-tpi.so’ -> ‘/usr/local/lib/libva-tpi.so.1.3904.0’
‘libva-wayland.so’ -> ‘/usr/local/lib/libva-wayland.so.1.3904.0’
‘libva-x11.so’ -> ‘/usr/local/lib/libva-x11.so.1.3904.0’
  • 进入到系统MESA库目录下

  • 下面文件中有一些是.org结尾是的系统的MESA库文,为保险起见保留了它

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    $cd /usr/lib/mesa-diverted/x86_64-linux-gnu
    $/usr/lib/mesa-diverted/x86_64-linux-gnu$ ls -l
    total 836
    lrwxrwxrwx 1 root root 15 Aug 20 2015 libEGL.so -> libEGL.so.1.0.0
    lrwxrwxrwx 1 root root 15 Aug 20 2015 libEGL.so.1 -> libEGL.so.1.0.0
    lrwxrwxrwx 1 root root 30 Aug 7 01:08 libEGL.so.1.0.0 -> /usr/local/lib/libEGL.so.1.0.0
    -rw-r--r-- 1 root root 173144 Aug 20 2015 libEGL.so.1.0.0.org
    lrwxrwxrwx 1 root root 21 Aug 20 2015 libGLESv1_CM.so -> libGLESv1_CM.so.1.1.0
    lrwxrwxrwx 1 root root 21 Aug 20 2015 libGLESv1_CM.so.1 -> libGLESv1_CM.so.1.1.0
    lrwxrwxrwx 1 root root 36 Aug 7 01:08 libGLESv1_CM.so.1.1.0 -> /usr/local/lib/libGLESv1_CM.so.1.1.0
    -rw-r--r-- 1 root root 18232 Aug 20 2015 libGLESv1_CM.so.1.1.0.org
    lrwxrwxrwx 1 root root 18 Aug 20 2015 libGLESv2.so -> libGLESv2.so.2.0.0
    lrwxrwxrwx 1 root root 18 Aug 20 2015 libGLESv2.so.2 -> libGLESv2.so.2.0.0
    lrwxrwxrwx 1 root root 33 Aug 7 01:10 libGLESv2.so.2.0.0 -> /usr/local/lib/libGLESv2.so.2.0.0
    -rw-r--r-- 1 root root 26424 Aug 20 2015 libGLESv2.so.2.0.0.org
    lrwxrwxrwx 1 root root 10 Aug 6 22:07 libGL.so -> libGL.so.1
    lrwxrwxrwx 1 root root 14 Aug 20 2015 libGL.so.1 -> libGL.so.1.2.0
    lrwxrwxrwx 1 root root 29 Aug 7 01:10 libGL.so.1.2.0 -> /usr/local/lib/libGL.so.1.2.0
    -rw-r--r-- 1 root root 627320 Aug 20 2015 libGL.so.1.2.0.org
  • 查看本机的显卡数量,我这台笔记本电脑有两个GPU如下:

    1
    2
    3
    $ lspci | grep "VGA"
    00:01.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] BeaverCreek [Radeon HD 6520G]
    01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Thames [Radeon HD 7500M/7600M Series]
  • 我这里修改/etc/X11/xorg.conf如下:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    118
    119
    120
    121
    122
    123
    124
    125
    126
    127
    128
    129
    130
    131
    132
    133
    134
    135
    136
    137
    $ grep -v '#|^$' /etc/X11/xorg.conf
    Section "ServerLayout"
    Identifier "X.org Configured"
    Screen 0 "Screen0" 0 0
    Screen 1 "Screen1" RightOf "Screen0"
    InputDevice "Mouse0" "CorePointer"
    InputDevice "Keyboard0" "CoreKeyboard"
    EndSection

    Section "Files"
    ModulePath "/usr/local/lib/xorg/modules"
    ModulePath "/usr/lib/xorg/modules"
    FontPath "/usr/share/fonts/X11/misc"
    FontPath "/usr/share/fonts/X11/cyrillic"
    FontPath "/usr/share/fonts/X11/100dpi/:unscaled"
    FontPath "/usr/share/fonts/X11/75dpi/:unscaled"
    FontPath "/usr/share/fonts/X11/Type1"
    FontPath "/usr/share/fonts/X11/100dpi"
    FontPath "/usr/share/fonts/X11/75dpi"
    FontPath "built-ins"
    EndSection

    Section "Module"
    Load "glx"
    EndSection

    Section "InputDevice"
    Identifier "Keyboard0"
    Driver "kbd"
    EndSection

    Section "InputDevice"
    Identifier "Mouse0"
    Driver "mouse"
    Option "Protocol" "auto"
    Option "Device" "/dev/input/mice"
    Option "ZAxisMapping" "4 5 6 7"
    EndSection

    Section "Monitor"
    Identifier "Monitor0"
    VendorName "Monitor Vendor"
    ModelName "Monitor Model"
    EndSection

    Section "Monitor"
    Identifier "Monitor1"
    VendorName "Monitor Vendor"
    ModelName "Monitor Model"
    EndSection


    Section "Device"
    ### Available Driver options are:-
    ### Values: <i>: integer, <f>: float, <bool>: "True"/"False",
    ### <string>: "String", <freq>: "<f> Hz/kHz/MHz",
    ### <percent>: "<f>%"
    ### [arg]: arg optional
    #Option "Accel" # [<bool>]
    #Option "SWcursor" # [<bool>]
    #Option "EnablePageFlip" # [<bool>]
    #Option "ColorTiling2D" # [<bool>]
    #Option "RenderAccel" # [<bool>]
    #Option "SubPixelOrder" # [<str>]
    #Option "AccelMethod" "glamor"
    Option "DRI" "3"
    Option "ColorTiling" "1"
    Option "AccelMethod" "EXA"
    Option "TearFree" "on"
    #Option "EnablePageFlip" "1"
    Option "RenderAccel" "on" # [<bool>]
    Option "AGPMode" "4"
    #Option "AIGLX" "off"
    #Option "EXAVSync" # [<bool>]
    #Option "EXAPixmaps" # [<bool>]
    #Option "ZaphodHeads" # <str>
    #Option "SwapbuffersWait" # [<bool>]
    Identifier "Advanced Micro Devices, Inc. [AMD/ATI] BeaverCreek [Radeon HD 6520G]"
    Driver "radeon"
    BusID "PCI:0:1:0"
    EndSection

    Section "Device"
    ### Available Driver options are:-
    ### Values: <i>: integer, <f>: float, <bool>: "True"/"False",
    ### <string>: "String", <freq>: "<f> Hz/kHz/MHz",
    ### <percent>: "<f>%"
    ### [arg]: arg optional
    #Option "Accel" # [<bool>]
    #Option "SWcursor" # [<bool>]
    #Option "EnablePageFlip" # [<bool>]
    #Option "ColorTiling" # [<bool>]
    #Option "ColorTiling2D" # [<bool>]
    Option "RenderAccel" "on" # [<bool>]
    Option "AGPMode" "4"
    #Option "SubPixelOrder" # [<str>]
    #Option "AccelMethod" # <str>
    #Option "EXAVSync" # [<bool>]
    #Option "EXAPixmaps" # [<bool>]
    #Option "ZaphodHeads" # <str>
    #Option "EnablePageFlip" # [<bool>]
    #Option "SwapbuffersWait" # [<bool>]
    #Option "AIGLX" "off"
    Option "DRI" "3"
    Option "ColorTiling" "1"
    Option "AccelMethod" "EXA"
    Option "TearFree" "on"
    #Option "EnablePageFlip" "1"
    Identifier "Advanced Micro Devices, Inc. [AMD/ATI] Thames [Radeon HD 7500M/7600M Series]"
    Driver "radeon"
    BusID "PCI:1:0:0"
    EndSection

    Section "Screen"
    Identifier "Screen0"
    Device "Card0"
    Monitor "Monitor0"
    SubSection "Display"
    Viewport 0 0
    Depth 24
    EndSubSection
    EndSection

    Section "Screen"
    Identifier "Screen1"
    Device "Card1"
    Monitor "Monitor1"
    SubSection "Display"
    Viewport 0 0
    Depth 24
    EndSubSection
    EndSection

    Section "Extensions"
    Option "Composite" "Enable"
    EndSection

  • 重启Xorg,用xrander 查看可用的GPU数量,下面显示两个GPU可供使用.

    1
    2
    3
    4
    $ xrandr --listproviders
    Providers: number : 2
    Provider 0: id: 0x8a cap: 0xf, Source Output, Sink Output, Source Offload, Sink Offload crtcs: 2 outputs: 3 associated providers: 0 name:radeon
    Provider 1: id: 0x55 cap: 0xf, Source Output, Sink Output, Source Offload, Sink Offload crtcs: 6 outputs: 0 associated providers: 0 name:radeon
  • 修改radeon驱动的内核参数,RadeonFeature

    1
    2
    3
    4
    5
    $ cat /etc/modprobe.d/radeon.conf
    options radeon audio=1
    options radeon agpmode=1
    options radeon tv=1
    options radeon modeset=1
  • 测试GLX信息, 如下显示GLX功能正常

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    $ glxinfo | head -n 10
    name of display: :0.0
    display: :0 screen: 0
    direct rendering: Yes
    server glx vendor string: SGI
    server glx version string: 1.4
    server glx extensions:
    GLX_ARB_create_context, GLX_ARB_create_context_profile,
    GLX_ARB_create_context_robustness, GLX_ARB_fbconfig_float,
    GLX_ARB_framebuffer_sRGB, GLX_ARB_multisample,
    GLX_EXT_create_context_es2_profile, GLX_EXT_framebuffer_sRGB,
    [...]
  • 检查VDPAU支持,如下显示已经开启VAPDU功能.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    $ vainfo
    libva info: VA-API version 0.36.0
    libva info: va_getDriverName() returns 0
    libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/r600_drv_video.so
    libva info: Found init function __vaDriverInit_0_35
    libva info: va_openDriver() returns 0
    vainfo: VA-API version: 0.36 (libva 1.4.1)
    vainfo: Driver version: Splitted-Desktop Systems VDPAU backend for VA-API - 0.7.4
    vainfo: Supported profile and entrypoints
    VAProfileMPEG2Simple : VAEntrypointVLD
    VAProfileMPEG2Main : VAEntrypointVLD
    VAProfileMPEG4Simple : VAEntrypointVLD
    VAProfileMPEG4AdvancedSimple : VAEntrypointVLD
    VAProfileH264Baseline : VAEntrypointVLD
    VAProfileH264Main : VAEntrypointVLD
    VAProfileH264High : VAEntrypointVLD
    VAProfileVC1Simple : VAEntrypointVLD
    VAProfileVC1Main : VAEntrypointVLD
    VAProfileVC1Advanced : VAEntrypointVLD

遇到过的名错误

  • 检查Xorg错误信息 ,如果出现很多”EE”信息,轻则功能缺失,重则不能开启图界面,下面只有一个EE,没有找到出错的原因.
    1
    2
    3
    $ grep "EE" /var/log/Xorg.0.log
    (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
    [ 6597.997] (EE) RADEON(1): No modes.

vainfo , vdpauinfo 出错.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
~$ VDPAU_DRIVER=r600 vdpauinfo
display: :0.0 screen: 0
Error creating VDPAU device: 23

~$ vdpauinfo
display: :0.0 screen: 0
Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory
Error creating VDPAU device: 1

~$ vainfo
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
vaInitialize failed with error code -1 (unknown libva error),exit

  • 上面错误是因为 /etc/X11/xorg.conf 的 AccelMethod 设置 Glamor ,改成EXA 解决问题.

新显卡加载固件问题

* R7 显卡遇无法加载radeon驱动

1
2
$ lspci | grep "VGA"
00:01.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Kaveri [Radeon R7 Graphics] (rev d6)
  • 通过dmesg 发现如下错误

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    [  449.155790] radeon 0000:00:01.0: Direct firmware load for
    radeon/kaveri_pfp.bin failed with error -2
    [ 449.156090] radeon 0000:00:01.0: Direct firmware load for
    radeon/kaveri_me.bin failed with error -2
    [ 449.156407] radeon 0000:00:01.0: Direct firmware load for
    radeon/kaveri_ce.bin failed with error -2
    [ 449.156680] radeon 0000:00:01.0: Direct firmware load for
    radeon/kaveri_mec.bin failed with error -2
    [ 449.156990] radeon 0000:00:01.0: Direct firmware load for
    radeon/kaveri_mec2.bin failed with error -2
    [ 449.156992] cik_cp: Failed to load firmware "radeon/kaveri_mec2.bin"
    [ 449.157153] [drm:cik_init [radeon]] *ERROR* Failed to load firmware!
    [ 449.157241] radeon 0000:00:01.0: Fatal error during GPU init
    [ 449.157327] [drm] radeon: finishing device.
    [ 449.164593] [TTM] Finalizing pool allocator
    [ 449.164597] [TTM] Finalizing DMA pool allocator
    [ 449.164697] [TTM] Zone kernel: Used memory at exit: 0 kiB
    [ 449.164701] [TTM] Zone dma32: Used memory at exit: 0 kiB
    [ 449.164703] [drm] radeon: ttm finalized
    [ 449.165170] radeon: probe of 0000:00:01.0 failed with error -2
  • 下载最新的固件 , 把radeon目录下的所有文件复制到/lib/firmware/radeon,重新modprobe radeon解决

  • 使用xrandr 设置多屏

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    $ xrandr
    Screen 0: minimum 320 x 200, current 3286 x 1080, maximum 8192 x 8192
    VGA-0 disconnected (normal left inverted right x axis y axis)
    LVDS connected 1366x768+0+0 (normal left inverted right x axis y axis) 344mm x 194mm
    1366x768 60.00*+
    1280x720 59.86
    1152x768 59.78
    1024x768 59.92
    800x600 59.86
    848x480 59.66
    720x480 59.71
    640x480 59.38
    HDMI-0 connected 1920x1080+1366+0 (normal left inverted right x axis y axis) 480mm x 270mm
    1920x1080 60.00*+ 50.00 59.94
    1920x1080i 60.00 50.00 59.94
    1680x1050 59.88
    1400x1050 59.95
    1600x900 60.00
    1280x1024 75.02 60.02
    1440x900 59.90
    1280x800 59.91
    1152x864 75.00
    1280x720 60.00 50.00 59.94
    1024x768 75.08 60.00
    800x600 75.00 60.32
    720x576 50.00
    720x480 60.00 59.94
    640x480 75.00 60.00 59.94
    720x400 70.08
    $ xrandr --output LVDS --mode 1366x768 --output HDMI-0 --mode 1920x1080 --right-of LVDS
  • 基于debian发行版安装mesa 驱并支持,VDPAU,VA API,GLX, 需要安装如下的包,下面是64位系统上的包列表.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# uname -a
Linux debian 4.8.13-20161211 #2 SMP Sun Dec 11 13:11:44 CST 2016 x86_64 GNU/Linux
# lspci | grep "VGA"
00:01.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Kaveri [Radeon R7 Graphics] (rev d6)


# apt-get install glx-alternative-mesa libegl1-mesa:amd64 libegl1-mesa-dev:amd64 libegl1-mesa-drivers:amd64 libgl1-mesa-dev:amd64 libgl1-mesa-dri:amd64
libgl1-mesa-dri:i386 libgl1-mesa-glx:amd64 libgl1-mesa-glx:i386 libglapi-mesa:amd64 libglapi-mesa:i386 libgles1-mesa:amd64
libgles1-mesa-dev:amd64 libgles2-mesa:amd64 libgles2-mesa-dev:amd64 libglu1-mesa:amd64 libglu1-mesa:i386
libglu1-mesa-dev libglw1-mesa:amd64 libglw1-mesa-dev libopenvg1-mesa:amd64 libosmesa6:amd64
libosmesa6-dev:amd64 libwayland-egl1-mesa:amd64 mesa-common-dev:amd64 mesa-opencl-icd
mesa-utils mesa-va-drivers:amd64 mesa-vdpau-drivers:amd64 libdrm-amdgpu1:amd64 libdrm-dev:amd64
libdrm-intel1:amd64 libdrm-intel1:i386 libdrm-nouveau2:amd64 libdrm-nouveau2:i386 libdrm-radeon1:amd64
libdrm-radeon1:i386 libdrm2:amd64 libdrm2:i386 libva-drm1:amd64 libvdpau1:amd64
mesa-vdpau-drivers:amd64 vdpauinfo libva-dev:amd64 libva-drm1:amd64 libva-egl1:amd64
libva-glx1:amd64 libva-tpi1:amd64 libva-wayland1:amd64 libva-x11-1:amd64 libva1:amd64
libva1:i386 libval-bin libval14:amd64


谢谢支持

- 参考链接

0x1 下载Raspberrypi 的工具链

1
2
$ git clone  https://github.com/raspberrypi/tools.git
$ cd tools
  • tools目录结构如下:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    ├── arm-bcm2708
    │   ├── arm-bcm2708hardfp-linux-gnueabi
    │   ├── arm-bcm2708-linux-gnueabi
    │   ├── arm-rpi-4.9.3-linux-gnueabihf
    │   ├── gcc-linaro-arm-linux-gnueabihf-raspbian
    │   └── gcc-linaro-arm-linux-gnueabihf-raspbian-x64
    ├── armstubs
    ├── configs
    ├── mkimage
    ├── pkg
    ├── test_code
    └── usbboot
  • 因为我的操作系统是debian x86_64 ,所以我要使用 gcc-linaro-arm-linux-gnueabihf-raspbian-x64 下面的工具链程序.设置环境变量

    1
    2
    3
    4
    5
    6
    ~$ export TOOLCHIAN_PATH=/home/user/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin
    ~$ export PATH=$PATH:$TOOLCHIAN_PATH
    ~$ export CROSS=arm-linux-gnueabihf-
    ~$ export CROSS_COMPILE=${CROSS}
    ~$ export CC=${CROSS}gcc
    ~$ export GCC=${CC}

CPUInfo

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
~$ cat /proc/cpuinfo
processor : 0
model name : ARMv6-compatible processor rev 7 (v6l)
BogoMIPS : 697.95
Features : half thumb fastmult vfp edsp java tls
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part : 0xb76
CPU revision : 7

Hardware : BCM2835
Revision : 000f
Serial : 000000007ea6137d

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
~ $ v4l2-ctl --list-devices

bcm2835-codec-decode (platform:bcm2835-codec):
/dev/video10
/dev/video11
/dev/video12

mmal service -833381080.-107286 (platform:bcm2835-v4l2):
/dev/video0

~ $ v4l2-ctl -D
Driver Info:
Driver name : bm2835 mmal
Card type : mmal service 0.-1087962660
Bus info : platform:bcm2835-v4l2
Driver version : 4.19.97
Capabilities : 0x85200005
Video Capture
Video Overlay
Read/Write
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x05200005
Video Capture
Video Overlay
Read/Write
Streaming
Extended Pix Format
~ $ v4l2-ctl -l

User Controls

brightness 0x00980900 (int) : min=0 max=100 step=1 default=50 value=50 flags=slider
contrast 0x00980901 (int) : min=-100 max=100 step=1 default=0 value=0 flags=slider
saturation 0x00980902 (int) : min=-100 max=100 step=1 default=0 value=0 flags=slider
red_balance 0x0098090e (int) : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
blue_balance 0x0098090f (int) : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
horizontal_flip 0x00980914 (bool) : default=0 value=0
vertical_flip 0x00980915 (bool) : default=0 value=0
power_line_frequency 0x00980918 (menu) : min=0 max=3 default=1 value=1
sharpness 0x0098091b (int) : min=-100 max=100 step=1 default=0 value=0 flags=slider
color_effects 0x0098091f (menu) : min=0 max=15 default=0 value=0
rotate 0x00980922 (int) : min=0 max=360 step=90 default=0 value=0 flags=modify-layout
color_effects_cbcr 0x0098092a (int) : min=0 max=65535 step=1 default=32896 value=32896

Codec Controls

video_bitrate_mode 0x009909ce (menu) : min=0 max=1 default=0 value=0 flags=update
video_bitrate 0x009909cf (int) : min=25000 max=25000000 step=25000 default=10000000 value=10000000
repeat_sequence_header 0x009909e2 (bool) : default=0 value=0
h264_i_frame_period 0x00990a66 (int) : min=0 max=2147483647 step=1 default=60 value=60
h264_level 0x00990a67 (menu) : min=0 max=11 default=11 value=11
h264_profile 0x00990a6b (menu) : min=0 max=4 default=4 value=4

Camera Controls

auto_exposure 0x009a0901 (menu) : min=0 max=3 default=0 value=0
exposure_time_absolute 0x009a0902 (int) : min=1 max=10000 step=1 default=1000 value=1000
exposure_dynamic_framerate 0x009a0903 (bool) : default=0 value=0
auto_exposure_bias 0x009a0913 (intmenu): min=0 max=24 default=12 value=12
white_balance_auto_preset 0x009a0914 (menu) : min=0 max=10 default=1 value=1
image_stabilization 0x009a0916 (bool) : default=0 value=0
iso_sensitivity 0x009a0917 (intmenu): min=0 max=4 default=0 value=0
iso_sensitivity_auto 0x009a0918 (menu) : min=0 max=1 default=1 value=1
exposure_metering_mode 0x009a0919 (menu) : min=0 max=2 default=0 value=0
scene_mode 0x009a091a (menu) : min=0 max=13 default=0 value=0

JPEG Compression Controls

compression_quality 0x009d0903 (int) : min=1 max=100 step=1 default=30 value=30

  • 根据查资料所知,设置GCC的安全的CFLAGS指令选项是-O2 -pipe -march=armv6 -mfpu=vfp -mfloat-abi=hard,还可以试一下这个-march=armv6zk -mtune=arm1176jzf-s -mfloat-abi=soft

0x2 编译ffmpeg依赖软件包

  • 下面先要编译安装一些依赖的库,这里用到的x264,libmp3lame,编译的过程出现任何问题,要查看config.log分析处理解决.假设编译安装的目是/home/user 具体可以自己更改.如果编译的东西非常多,会需一个个按依赖顺序逐个编译,如果动态库还要考虑与原系统里的库是ABI兼容的,GCC的版本也是要统一的,如果编译成静态库可以考虑使用该方法.
  • 也可以用另一个方法,在RPi安装好相关的一些库如:sudo apt-get install libssl1.0-dev libbz2-1.0 libmbedcrypto3 libmbedtls-dev nettle-dev libbz2-dev,安装完成之后,把RPi里的系统卡挂载到编译机上面去,指定--sysroot为这个RPi的根系统顶层目录,本文的RPi使用的系统是Raspbian 10,挂载的位置是/media/michael/rootfs,所以要指定--sysroot=/media/michael/rootfs,编译完成后安装路径可以指定为--prefix=/media/michael/root/usr/local.
  • 如果直接使用一张安装了RPi的SD卡挂载做交叉编译的sysroot,需要处理卡里文件系统的一些绝对引用链接文件,它在RPi系统理正常,具体可以参考这里,ffmpeg编译配置前检测到一些系统库的链接是broken,就会认为是没有这个库,在实际会出现ffmpeg无法开启pthread的特性支持.下面是处理它的脚本.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
~$ cat symlinks.sh
#!/bin/bash
cd /media/michael/rootfs/usr/lib/arm-linux-gnueabihf

for L in $(find -type l)
do
ABS=$(readlink $L)
if [[ $ABS =~ ^/ ]]
then
# 这要根据实际情况来改.
# 原来 : /usr/lib/arm-linux-gnueabihf/libanl.so -> /lib/arm-linux-gnueabihf/libanl.so.1
# 修改后 : /usr/lib/arm-linux-gnueabihf/libanl.so -> ../../../lib/arm-linux-gnueabihf/libanl.so.1
RELPATH=$(realpath --relative-to=$L -s ../../$ABS)
echo fixing link of $L to $ABS with $RELPATH
sudo ln -svf $RELPATH $L
fi
done

x264

1
2
3
4
5
~$  git clone http://git.videolan.org/git/x264.git
~$ cd x264 && mkdir build-rpi && cd build-rpi
~$ ../configure --enable-static --enable-shared --cross-prefix=arm-linux-gnueabihf- --host=arm-linux \
--enable-strip --enable-pic --extra-cflags='-O2 -pipe -march=armv6 -mfpu=vfp -mfloat-abi=hard' --prefix=/home/user/
~$ make && make install
  • 如果出现下面错误,禁用--disable-opencl特性.
1
../common/opencl.c:116:27: fatal error: common/oclobj.h: No such file or directory

libmp3lame

1
2
3
4
$ git clone https://github.com/gypified/libmp3lame
$ cd libmp3lame && mkdir build-rpi && build-rpi
$ ../configure --host=arm-linux --prefix=/home/user
$ make && make install
  • 编译完成后的/home/user 目录结构如下:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    $ tree user
    user
    ├── bin
    │   ├── lame
    │   └── x264
    ├── include
    │   ├── lame
    │   │   └── lame.h
    │   ├── x264_config.h
    │   └── x264.h
    ├── lib
    │   ├── libmp3lame.a
    │   ├── libmp3lame.la
    │   ├── libmp3lame.so -> libmp3lame.so.0.0.0
    │   ├── libmp3lame.so.0 -> libmp3lame.so.0.0.0
    │   ├── libmp3lame.so.0.0.0
    │   ├── libx264.a
    │   ├── libx264.so -> libx264.so.148
    │   ├── libx264.so.148
    │   └── pkgconfig
    │   └── x264.pc

安装libvpx

安装 yasm

  • 如果RPi没有安装yasm,这里就要编译一个,libvpx要用到它,还有其它一些编解库可能要用到它.
1
2
3
4
5
6
7
8
9
~$  wget -c https://www.tortall.net/projects/yasm/releases/yasm-1.3.0.tar.gz
~$ cd yasm-1.3 && mkdir build-rpi && cd build-rpi
~$ CROSS=arm-linux-gnueabihf- \
DEFAULT_INCLUDES="-I/media/michael/rootfs/usr/include/arm-linux-gnueabihf -I/media/michael/rootfs/usr/include" \
CFLAGS="-O2 -pipe -march=armv6 -mfpu=vfp -mfloat-abi=hard" \
LDFLAGS="--sysroot=/media/michael/rootfs -L/media/michael/rootfs/usr/lib/arm-linux-gnueabihf" \
../configure --host=arm-linux-gnueabihf --disable-debug --disable-python \
--prefix=/media/michael/rootfs/usr/local
~$ make && make install
  • 编译安装libvpx
1
2
3
4
5
6
7
8
9
10
11
12
~$ git clone https://chromium.googlesource.com/webm/libvpx
~$ cd libvpx && mkdir build-rpi && cd build-rpi
~$ CROSS=arm-linux-gnueabihf- \
CFLAGS="-I/media/michael/rootfs/usr/include -O2 -pipe -march=armv6 -mfpu=vfp -mfloat-abi=hard \
-I/media/michael/rootfs/usr/include/arm-linux-gnueabihf" \
LDFLAGS="--sysroot=/media/michael/rootfs -L/media/michael/rootfs/usr/lib/arm-linux-gnueabihf" \
../configure --target=armv7-linux-gcc --disable-examples --disable-docs --enable-vp9-highbitdepth
--enable-better-hw-compatibility --enable-vp8 --enable-vp9 \
--enable-postproc --enable-vp9-postproc --disable-multithread --enable-realtime-only --enable-shared --enable-runtime-cpu-detect
--enable-webm-io --enable-libyuv --prefix=/media/michael/rootfs/usr/local \
--enable-multi-res-encoding --enable-internal-stats --as=yasm
~$ make && make install

Opus音频

1
2
3
4
5
6
7
8
9
10
~$ git clone https://github.com/xiph/opus.git
~$ cd opus && ./autogen.sh && mkdir build-rpi && cd build-rpi
~$ CROSS=arm-linux-gnueabihf- \
CFLAGS="-I/media/michael/rootfs/usr/include -O2 -pipe -march=armv6 -mfpu=vfp -mfloat-abi=hard \
-I/media/michael/rootfs/usr/local/include -I/media/michael/rootfs/usr/include \
-I/media/michael/rootfs/usr/include/arm-linux-gnueabihf " \
LDFLAGS="--sysroot=/media/michael/rootfs -L/media/michael/rootfs/usr/local/lib -L/media/michael/rootfs/usr/lib" \
../configure --host=arm-linux-gnueabihf --disable-doc --enable-check-asm --enable-float-approx \
--with-sysroot=/media/michael/rootfs --prefix=/media/michael/rootfs/usr/local
~$ make && make install

安装zlib

1
2
3
4
~$ wget -c https://www.zlib.net/zlib-1.2.11.tar.gz
~$ tar xvf zlib-1.2.11.tar.gz && cd zlib-1.2.11 && mkdir build-rpi
~$ cd build-rpi && ../configure --prefix=/home/user
~$ make && make install

RTMP支持

  • rtmpdump这个库有很久没有更新了,如果要使用加密功能,需一些加密库来支持,且它的编译不像是上面的x264,zlib那样,它需要指定--sysroot才能正确的编译与链接原系统的库.
1
~$ git clone https://git.ffmpeg.org/rtmpdump.git

OpenSSL

1
2
3
4
5
6
7
8
9
10
~$ wget -c https://www.openssl.org/source/openssl-1.1.1d.tar.gz
~$ tar xvf openssl-1.1.1d.tar.gz && cd openssl-1.1.1d
~$ export CROSS=arm-linux-gnueabihf
~$ export AR=${CROSS}-ar
~$ export AS=${CROSS}-as
~$ export CC=${CROSS}-gcc
~$ export CXX=${CROSS}-g++
~$ export LD=${CROSS}-ld
~$ ./Configure linux-armv4 --prefix=/home/user --openssldir=/home/user/openssl
~$ make && make install
  • 编译使用OpenSSL支持
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
~$ cd rtmpdump/librtmp
~$ make clean && make CROSS_COMPILE=arm-linux-gnueabihf- \
INC="-I/media/michael/rootfs/usr/include -I/media/michael/rootfs/usr/include/arm-linux-gnueabihf" \
LDFLAGS="--sysroot=/media/michael/rootfs -L/media/michael/rootfs/usr/lib/arm-linux-gnueabihf" \
CFLAGS="-O2 -pipe -march=armv6 -mfpu=vfp -mfloat-abi=hard"
rm -f *.o *.a *.so *.so.1 librtmp.pc
arm-linux-gnueabihf-gcc -Wall -I/media/michael/rootfs/usr/include -I/media/michael/rootfs/usr/include/arm-linux-gnueabihf -DRTMPDUMP_VERSION=\"v2.4\" -DUSE_OPENSSL -O2 -fPIC -c -o rtmp.o rtmp.c
rtmp.c: In function ‘RTMP_ReadPacket’:
rtmp.c:3555:7: warning: variable ‘didAlloc’ set but not used [-Wunused-but-set-variable]
int didAlloc = FALSE;
^
arm-linux-gnueabihf-gcc -Wall -I/media/michael/rootfs/usr/include -I/media/michael/rootfs/usr/include/arm-linux-gnueabihf
-DRTMPDUMP_VERSION=\"v2.4\" -DUSE_OPENSSL -O2 -fPIC -c -o log.o log.c
arm-linux-gnueabihf-gcc -Wall -I/media/michael/rootfs/usr/include -I/media/michael/rootfs/usr/include/arm-linux-gnueabihf
-DRTMPDUMP_VERSION=\"v2.4\" -DUSE_OPENSSL -O2 -fPIC -c -o amf.o amf.c
arm-linux-gnueabihf-gcc -Wall -I/media/michael/rootfs/usr/include -I/media/michael/rootfs/usr/include/arm-linux-gnueabihf
-DRTMPDUMP_VERSION=\"v2.4\" -DUSE_OPENSSL -O2 -fPIC -c -o hashswf.o hashswf.c
arm-linux-gnueabihf-gcc -Wall -I/media/michael/rootfs/usr/include -I/media/michael/rootfs/usr/include/arm-linux-gnueabihf
-DRTMPDUMP_VERSION=\"v2.4\" -DUSE_OPENSSL -O2 -fPIC -c -o parseurl.o parseurl.c
arm-linux-gnueabihf-ar rs librtmp.a rtmp.o log.o amf.o hashswf.o parseurl.o
arm-linux-gnueabihf-ar: creating librtmp.a
arm-linux-gnueabihf-gcc -shared -Wl,-soname,librtmp.so.1 -Wl,-rpath-link,/media/michael/rootfs/usr/lib/arm-linux-gnueabihf
--sysroot=/media/michael/rootfs -L/media/michael/rootfs/usr/lib/arm-linux-gnueabihf -o librtmp.so.1 rtmp.o log.o amf.o hashswf.o
parseurl.o -lssl -lcrypto -lz
ln -sf librtmp.so.1 librtmp.so

GnuTLS

  • 使用GnuTLS来编译,需要在RPi里安装libgnutls28-dev libgmp-dev nettle-dev
1
2
3
4
5
6
7
~$ cd rtmpdump/librtmp
# 在Makefile里的第一行加入或修改prefix变量,可以改成/home/user也可以指向/media/michael/rootfs/usr/local .
~$ make clean && make CFLAGS="-O2 -pipe -march=armv6 -mfpu=vfp -mfloat-abi=hard" \
CROSS_COMPILE=arm-linux-gnueabihf- \
INC="-I/media/michael/rootfs/usr/include -I/media/michael/rootfs/usr/inculde -I/media/michael/rootfs/usr/include/arm-linux-gnueabihf" \
LDFLAGS="--sysroot=/media/michael/rootfs -L/media/michael/rootfs/usr/lib/arm-linux-gnueabihf" \
CRYPTO=GNUTLS SHARED=yes
  • 编译出错,因为使用了gcc-linaro-arm-linux-gnueabihf-raspbian-x64,编译主机是64位.需要安装libc6-dev-armhf-cross
1
2
3
4
5
6
7
arm-linux-gnueabihf-gcc -Wall  -I/media/michael/rootfs/usr/include -DRTMPDUMP_VERSION=\"v2.4\" -DUSE_OPENSSL  -O2 -fPIC   -c -o rtmp.o rtmp.c
In file included from rtmp.c:26:0:
/media/michael/rootfs/usr/include/stdint.h:26:36: fatal error: bits/libc-header-start.h: No such file or directory
#include <bits/libc-header-start.h>
^
compilation terminated.

  • 交叉编译时一定要指--sysroot选项,不然出下面的动态库的链接错误,具体可以打开系统下的/usr/lib/arm-linux-gnueabihf/libc.so查看.
1
2
3
4
5
6
7
8
9
10
arm-linux-gnueabihf-gcc -shared -Wl,-soname,librtmp.so.1 -L/media/michael/rootfs/usr/lib/arm-linux-gnueabihf -o librtmp.so.1
rtmp.o log.o amf.o hashswf.o parseurl.o -lgnutls -lhogweed -lnettle -lgmp -lz
/home/michael/3TB-DISK/github/raspberrypi-chaintools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/../lib/gcc/
arm-linux-gnueabihf/4.8.3/../../../../arm-linux-gnueabihf/bin/ld: cannot find /lib/arm-linux-gnueabihf/libc.so.6
/home/michael/3TB-DISK/github/raspberrypi-chaintools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/../lib/gcc/
arm-linux-gnueabihf/4.8.3/../../../../arm-linux-gnueabihf/bin/ld: cannot find /usr/lib/arm-linux-gnueabihf/libc_nonshared.a
/home/michael/3TB-DISK/github/raspberrypi-chaintools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/../lib/gcc/
arm-linux-gnueabihf/4.8.3/../../../../arm-linux-gnueabihf/bin/ld: cannot find /lib/arm-linux-gnueabihf/ld-linux-armhf.so.3
collect2: error: ld returned 1 exit status
make: *** [Makefile:95: librtmp.so.1] Error 1

0x3 下载并编译FFMPEG

硬件加速相关

  • --enable-decoder=h264_mmal --enable-mmal是需要mmal.h头文件的,--enable-encoder=h264_omx --enable-omx --enable-omx-rpi
    这些是要Openmax的头文件的.原本以为要下载OpenMAX IL头文件,后来发现这一步是不需要的,所有硬件开发的支持文件在RPi里的/opt/vc下面.

  • 如需开启--enable-opengl,需要在RPi安装gl库,如:libopengl-dev libgles-dev libgl-dev libegl-dev libopengl0 libgles2 libgles1.

  • 如果开启drawtext过滤器,也就是往视频上写字,需要加上--enable-libfreetype --enable-libfontconf --enable-libfribidi选项,要先行安装
    libfribidi-dev libfontconfig1-dev libfontconfig1的运行库与头文件.

  • 这里最好新建一个目录用来编译,防止在当前目录下编译污染了当前目录.

  • ffmpeg仓库发现,release/3.4之后无法支持omx硬件加速功能,4.2之后会报WARNING: Disabled h264_omx_encoder because not all dependencies are satisfied: omx的提示警告.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
~$ git clone https://git.ffmpeg.org/ffmpeg.git
~$ cd ffmpeg && mkdir build-rpi && cd build-rpi
~$ export ROOTFS=/media/michael/rootfs
~$ export PKG_CONFIG_PATH=${ROOTFS}/opt/vc/lib/pkgconfig:${ROOTFS}/usr/lib/pkgconfig:${ROOTFS}/usr/lib/arm-linux-gnueabihf/pkgconfig:${ROOTFS}/usr/local/lib/pkgconfig
~$ ./configure --arch=armhf --target-os=linux --disable-doc --disable-htmlpages --disable-manpages --disable-podpages \
--disable-txtpages --enable-librtmp --enable-zlib --enable-bzlib --enable-openssl --enable-shared --enable-static \
--enable-gpl --enable-libx264 --enable-omx --enable-omx-rpi --enable-mmal --enable-nonfree --enable-libopus \
--enable-libfreetype --enable-libfontconfig --enable-libfribidi \
--enable-runtime-cpudetect --enable-postproc --enable-pic --enable-neon --enable-vfp --enable-libvpx \
--enable-cross-compile --cross-prefix=arm-linux-gnueabihf- --enable-libmp3lame --enable-hardcoded-tables --disable-encoders \
--enable-encoder="opus,libopus,libvpx8,libvpx9,vp8_v4l2m2m,h264_omx,mjpeg,mpeg2video,png,libx264,h264_v4l2m2m,flv,mpeg4_v4l2m2m,mpeg4" \
--disable-decoders --enable-decoder="opus,libopus,libvpx8,libvpx9,vp9,vp9_v4l2m2m,vp8,vp8_v4l2m2m,png,mjpeg,h264,mpeg2video,mpeg2_mmal,h264_mmal,rawvideo,flv,mpeg4_mmal,mpeg4_v4l2m2m,vc1_mmal" \
--disable-muxers --enable-muxer="opus,webm,webm_chunk,webm_dash_manifest,dash,avi,mp4,hls,rtp_mpegts,rtsp,mpegts,mjpeg,h264,flv" \
--disable-demuxers --enable-demuxer="dash,webm_dash_manifest,h264,hls,mjpeg,rtp,rtsp,flv" \
--disable-parsers --enable-parser="opus,vp8,vp9,png,h264,mjpeg,mpeg4video" \
--sysroot=/media/michael/rootfs \
--extra-cflags="-I/media/michael/rootfs/usr/include -I/media/michael/rootfs/opt/vc/include/IL \
-I/media/michael/rootfs/opt/vc/include -I/media/michael/rootfs/usr/include/arm-linux-gnueabihf \
-I/media/michael/rootfs/usr/include/freetype2 -I/media/michael/rootfs/usr/include/fribidi \
-O2 -pipe -march=armv6 -mfpu=vfp -mfloat-abi=hard" \
--extra-cxxflags='-O2 -pipe -march=armv6 -mfpu=vfp -mfloat-abi=hard' \
--extra-ldflags="--sysroot=/media/michael/rootfs -L/media/michael/rootfs/opt/vc/lib \
-L/media/michael/rootfs/usr/lib/arm-linux-gnueabihf -L/media/michael/rootfs/opt/vc/lib \
-L/media/michael/rootfs/usr/lib -L/media/michael/rootfs/lib/arm-linux-gnueabihf -lmmal -lmmal_core -lvcos
-lmmal_vc_client -lmmal_components -lvchiq_arm \
-lvcsm -lmmal_util -lcontainers -lpthread" \
--extra-libs="-lz -lrt -lresolv -lnsl -lm -ldl -lbz2 -lpthread" \
--prefix=/media/michael/rootfs/usr/local

install prefix /media/michael/rootfs/usr/local
source path .
C compiler arm-linux-gnueabihf-gcc
C library glibc
host C compiler gcc
host C library glibc
ARCH arm (armv6)
big-endian no
runtime cpu detection yes
ARMv5TE enabled yes
ARMv6 enabled yes
ARMv6T2 enabled yes
VFP enabled yes
NEON enabled yes
THUMB enabled no
debug symbols yes
strip symbols yes
optimize for size no
optimizations yes
static yes
shared yes
postprocessing support yes
network support yes
threading support pthreads
safe bitstream reader yes
texi2html enabled no
perl enabled yes
pod2man enabled yes
makeinfo enabled yes
makeinfo supports HTML yes

External libraries:
bzlib libmp3lame libvpx libx264 openssl xlib zlib
iconv librtmp

External libraries providing hardware acceleration:
mmal omx v4l2_m2m

Libraries:
avcodec avfilter avformat avutil postproc swresample swscale
avdevice

Programs:
ffmpeg ffprobe ffserver

Enabled decoders:
flv h264_mmal mpeg2video mpeg4_v4l2m2m rawvideo vp8 vp9
h263 mjpeg mpeg4_mmal png vc1_mmal vp8_v4l2m2m vp9_v4l2m2m
h264 mpeg2_mmal

Enabled encoders:
flv h264_omx libx264 mpeg2video mpeg4_v4l2m2m png vp8_v4l2m2m
h263 h264_v4l2m2m mjpeg mpeg4

Enabled hwaccels:
h264_mmal mpeg2_mmal mpeg4_mmal vc1_mmal

Enabled parsers:
h263 h264 mjpeg mpeg4video png vp8 vp9

Enabled demuxers:
asf h264 matroska mov rm rtsp webm_dash_manifest
flv hls mjpeg mpegts rtp sdp

Enabled muxers:
adts ffm hls mov rtp rtsp webm_chunk
avi flv latm mp4 rtp_mpegts webm webm_dash_manifest
dash h264 mjpeg mpegts

Enabled protocols:
async file httpproxy librtmps mmst srtp tls_openssl
cache ftp https librtmpt pipe subfile udp
concat gopher icecast librtmpte prompeg tcp udplite
crypto hls librtmp md5 rtp tee unix
data http librtmpe mmsh

Enabled filters:
[....]

Enabled bsfs:
aac_adtstoasc dump_extradata hevc_mp4toannexb mjpega_dump_header mpeg4_unpack_bframes remove_extradata vp9_superframe
chomp extract_extradata imx_dump_header mov2textsub noise text2movsub vp9_superframe_split
dca_core h264_mp4toannexb mjpeg2jpeg mp3_header_decompress null vp9_raw_reorder

Enabled indevs:
fbdev lavfi oss v4l2

Enabled outdevs:
fbdev oss v4l2

License: nonfree and unredistributable
Creating configuration files ...
config.h is unchanged
libavutil/avconfig.h is unchanged

  • 如果不出问题出问题会在编译目录下出现以下二进制文件:ffserver,ffprobe,ffmpeg ,使用file 测试一下文件类型如下:
    1
    2
    3
    $ file ffmpeg
    ffmpeg: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked,
    interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.26, BuildID[sha1]=ec3417b6de5c9d89fe351048f6718a0857e760ff, stripped
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
~ $ ffmpeg -f v4l2 -list_formats all -i /dev/video0
[....]
[video4linux2,v4l2 @ 0x3e38b0] Raw : yuv420p : Planar YUV 4:2:0 : {32-2592, 2}x{32-1944, 2}
[video4linux2,v4l2 @ 0x3e38b0] Raw : yuyv422 : YUYV 4:2:2 : {32-2592, 2}x{32-1944, 2}
[video4linux2,v4l2 @ 0x3e38b0] Raw : rgb24 : 24-bit RGB 8-8-8 : {32-2592, 2}x{32-1944, 2}
[video4linux2,v4l2 @ 0x3e38b0] Compressed: mjpeg : JFIF JPEG : {32-2592, 2}x{32-1944, 2}
[video4linux2,v4l2 @ 0x3e38b0] Compressed: h264 : H.264 : {32-2592, 2}x{32-1944, 2}
[video4linux2,v4l2 @ 0x3e38b0] Compressed: mjpeg : Motion-JPEG : {32-2592, 2}x{32-1944, 2}
[video4linux2,v4l2 @ 0x3e38b0] Raw : Unsupported : YVYU 4:2:2 : {32-2592, 2}x{32-1944, 2}
[video4linux2,v4l2 @ 0x3e38b0] Raw : Unsupported : VYUY 4:2:2 : {32-2592, 2}x{32-1944, 2}
[video4linux2,v4l2 @ 0x3e38b0] Raw : uyvy422 : UYVY 4:2:2 : {32-2592, 2}x{32-1944, 2}
[video4linux2,v4l2 @ 0x3e38b0] Raw : nv12 : Y/CbCr 4:2:0 : {32-2592, 2}x{32-1944, 2}
[video4linux2,v4l2 @ 0x3e38b0] Raw : bgr24 : 24-bit BGR 8-8-8 : {32-2592, 2}x{32-1944, 2}
[video4linux2,v4l2 @ 0x3e38b0] Raw : yuv420p : Planar YVU 4:2:0 : {32-2592, 2}x{32-1944, 2}
[video4linux2,v4l2 @ 0x3e38b0] Raw : Unsupported : Y/CrCb 4:2:0 : {32-2592, 2}x{32-1944, 2}
[video4linux2,v4l2 @ 0x3e38b0] Raw : bgr0 : 32-bit BGRA/X 8-8-8-8 : {32-2592, 2}x{32-1944, 2}

  • 把上述三个文件复制到树莓派的机器下就可以直接运行了.

0x4 安装Web服务

Nginx

  • 可以安装现有的发行版软件也可以自己编译,这里就使系统安装使用Nginx做为Web服务.
1
~$ sudo apt-get install nginx libnginx-mod-rtmp

配置RTMP

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
~$ sudo cat > '/etc/nginx/conf.d/rtmp' << EOF
# RTMP media server
rtmp {
server {
listen 1935;
chunk_size 4096;
allow play all;
# Transcoding (ffmpeg needed)
# 这里纯流文件直播,使用ffmpeg 切片好的文件目录.
application play {
live on;
hls on;
record off;
hls_path /var/www/html/live/play;
}

application hls {
live on;
record off;

hls on;
hls_path /var/www/html/live/hls;
hls_nested on;
hls_type live;
hls_fragment 2s;
hls_variant _low BANDWIDTH=640000;
hls_variant _hi BANDWIDTH=2140000;
hls_cleanup on;
}

application dash {
live on;
record off;
dash on;
dash_path /var/www/html/live/dash;
dash_fragment 2s;
dash_nested on;
dash_playlist_length 20;
dash_cleanup on;
}
exec_static /var/www/html/ffmpeg.sh;
}
}
EOF

~$ sudo sh -c "echo \"include /etc/nginx/conf.d/rtmp;\" >> /etc/nginx/nginx.conf"
~$ id www-data
uid=33(www-data) gid=33(www-data) groups=33(www-data),44(video)
  • 如果没有像上面一样显示,www-datavideo里,可以直接修改/etc/group或用下面命令.
1
~$ sudo useradd -a -G video www-data
  • 使用shm共享内存的方式来暂存hls流片段.
1
2
3
~$ sudo mkdir -p /var/www/html/live/
~$ cd /var/www/html/live/
~$ sudo ln -s /dev/shm hls

开启Basic认证

1
2
3
~$ printf "username:$(openssl passwd -crypt your_passwd)\n" > conf.d/.htpasswd
~$ sudo chown www-data.root conf.d/.htpasswd
~$ sudo chmod 400 conf.d/.htpasswd
  • 修改nginx.conf里面的主机配置文件,包含如下行:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
     location /
    {
    auth_basic "Webcam 登录";
    auth_basic_user_file conf.d/.htpasswd;
    autoindex on;
    /* 如果是整个服务器都使用这一个密码,就把上面三行加到 server 块里 */
    }

    # 这里对应上面的 application play.
    location /play {
    root /var/www/html ;
    types {
    application/vnd.apple.mpegurl m3u8;
    video/mp2t ts;
    }
    add_header Cache-Control no-cache;

    # To avoid issues with cross-domain HTTP requests (e.g. during development)
    add_header Access-Control-Allow-Origin *;
    }
  • 访问测试,curl http://<username>:<password>@<server_ipaddress>/

推流脚本(HLS,Dash)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
~ $ cat ffmpeg.sh
FONT='/usr/share/fonts/truetype/wqy/wqy-zenhei.ttc'
PUB="transpose=1,format=yuv420p,"
VF1="drawtext=fontfile='${FONT}':text='时间\: %{localtime}':x=20:y=620:box=1:boxcolor=black@0.1:fontsize=28:fontcolor=white,"
VF2="drawtext=fontfile='${FONT}':textfile='/var/www/html/cpuinfo':x=2:y=2:boxcolor=white@0.1:box=1,"
VF3="drawtext=fontfile='${FONT}':textfile='/var/www/html/ffmpeg.sh':x=2:y=250:boxcolor=white@0.1:box=1,"
VF4="drawtext=fontfile='${FONT}':text='TCR\:':timecode=' 00\:00\:00\:00':boxcolor=black@0.1:fontsize=24:fontcolor=white:timecode_rate=25:x=20:y=650,"
H="drawtext=fontfile='${FONT}':text='HLS流直播':x=20:y=680:box=1:boxcolor=black@0.1:fontsize=28:fontcolor=white"
D="drawtext=fontfile='${FONT}':text='DASH流直播':x=20:y=680:box=1:boxcolor=black@0.1:fontsize=28:fontcolor=white"
VF=`echo ${PUB}${VF1}${VF2}${VF3}${VF4}`
HLS=`echo ${VF}${H}`
DASH=`echo ${VF}${D}`
/usr/local/bin/ffmpeg -hide_banner -loglevel error -thread_queue_size 2048 -f v4l2 \
-input_format yuv420p -re -i /dev/video0 -framerate 25 \
-c:v h264_omx -keyint_min 0 -map 0:v -b:v 1M -vf "${HLS}" \
-g 60 -an -f flv -rtmp_live live rtmp://127.0.0.2/hls/ \
-c:v h264_omx -keyint_min 0 -map 0:v -b:v 1M -vf "${DASH}" \
-g 60 -an -f flv -rtmp_live live rtmp://127.0.0.2/dash/


  • h264_v4l2m2m很流畅
1
ffmpeg -f v4l2 -video_size 1280x720 -i /dev/video0 -codec:v h264_v4l2m2m -b:v 2048k -f matroska test1.mkv

Systemd服务

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[Unit]
Description=ffmpeg listening and forward stream
After=syslog.target network.target

[Service]
PIDFile=/tmp/ffstreamer.pid
ExecStart= /path/to/ffmpeg.sh

ExecStop=/bin/kill -s QUIT $MAINPID
Restart=always
User=www-data

[Install]
WantedBy=multi-user.target
  • 简单HTML包装
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
<head>
<meta charset="utf-8"/>
<script src="https://cdn.jsdelivr.net/npm/hls.js@latest"></script>
<script src="http://cdn.dashjs.org/latest/dash.all.min.js"></script>
</head>
<body>
<h1>Raspberry Pi </h1>
<h2>Fmpeg HLS 直播流服务</h2>
<video id="video" autoplay controls height="1024" width="768"></video>
<h2> Fmpeg Dash 直播流服务</h2>
<video data-dashjs-player src="live/dash/index.mpd" autoplay controls height="1024" width="768"></video>
<script>
if(Hls.isSupported()) {
var video = document.getElementById('video');
var hls = new Hls();
hls.loadSource('live/hls/index.m3u8');
hls.attachMedia(video);
hls.on(Hls.Events.MANIFEST_PARSED,function() {
video.play();
});
}
</script>
<div>
<img src="rpd.jpg" />
</div>
</body>
</html>

编译Libwebsockets,mosquitto支持IPv6

  • 在RPI里可以直接安装发行版的mosquitto,libwebsockets库与服务,但是发现在发行版里的libwebsockets没有开启IPv6支持,导致mosquitto里的websockets协议服务,只有IPv4的服务,编译安装仅仅只为了让它支持IPv6.

libuv编译

1
2
3
4
5
6
7
8
9
~$ git clone https://github.com/libuv/libuv.git
# 初始化交叉编译环境变量,确定安装了automake,libtool.
~$ ./autogen.sh && mkdir build-rpi && cd build-rpi
~$ ../configure --host=arm-linux-gnueabihf --prefix=/media/michael/rootfs/usr/local
# 下面sudo的环境,也要指定交叉工具链的绝对路径.
~$ make && sudo PATH=$PATH:$TOOLCHIAN_PATH make install
~$ file /media/michael/rootfs/usr/local/lib/libuv.so.1.0.0
/media/michael/rootfs/usr/local/lib/libuv.so.1.0.0: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=431be81e472cf960d64fdd030254e88e5828d05c, with debug_info, not stripped

libwebsockets编译

1
~$ sudo apt-get install libmbedtls-dev  libmbedtls12 libssl-dev -y
1
2
3
4
5
6
7
8
9
~$ git clone https://libwebsockets.org/repo/libwebsockets
~$ cd libwebsockets && mkdir build-rpi
~$ export ROOTFS=/media/michael/rootfs
~$ cmake -DCMAKE_SYSROOT=$ROOTFS -DCMAKE_INSTALL_PREFIX:PATH=$ROOTFS/usr/local \
-DCMAKE_TOOLCHAIN_FILE=../contrib/cross-arm-linux-gnueabihf.cmake -DLWS_WITH_LWSWS=1 -DCMAKE_BUILD_TYPE=Release \
-DLWS_WITH_ZLIB=1 -DLWS_IPV6=1 -DLWS_WITH_SOCKS5=1 -DLWS_WITH_HTTP2=1 -DLWS_ROLE_WS=1 -DLWS_WITH_SYS_ASYNC_DNS=1 \
-DLWS_WITH_HTTP_BASIC_AUTH=1 -DCMAKE_FIND_ROOT_PATH=$ROOTFS -DLWS_WITHOUT_TESTAPPS=1 -DLWS_WITH_MBEDTLS=1 \
-DCMAKE_C_FLAGS="-I$ROOTFS/usr/include -I$ROOTFS/usr/local/include -O2 -pipe -march=armv6 -mfpu=vfp -mfloat-abi=hard -I/$ROOTFS/usr/include/arm-linux-gnueabihf" ../

  • 错误

  • 如下面错误所示,就是有一个变量没有初始化,可能是这个GCC版本也没有相应处理机制,所以报错了,打开源码找到出错位置,给变量设置一个初值.

1
2
3
4
5
6
7
8
9
/fullpath/libwebsockets/lib/core-net/sorted-usec-list.c: In function ‘__lws_sul_service_ripe’:
/fullpath/libwebsockets/lib/core-net/sorted-usec-list.c:139:4: error: ‘lowest’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
return lowest - usnow;
^
cc1: all warnings being treated as errors
make[2]: *** [lib/CMakeFiles/websockets_shared.dir/build.make:869: lib/CMakeFiles/websockets_shared.dir/core-net/sorted-usec-list.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1076: lib/CMakeFiles/websockets_shared.dir/all] Error 2
make: *** [Makefile:163: all] Error 2

  • 编译Mosquitto
1
2
3
4
5
6
7
8
~$ git clone https://github.com/eclipse/mosquitto
~$ cd mosquitto && mkdir build-rpi
~$ export ROOTFS=/media/michael/rootfs
~$ cmake -DCMAKE_TOOLCHAIN_FILE=../../libwebsockets/contrib/cross-arm-linux-gnueabihf.cmake -DWITH_WEBSOCKETS=yes \
-DWITH_TLS=yes -DCMAKE_INSTALL_PREFIX:PATH=$ROOTFS/usr/local -DWITH_DOCS=no -DDOCUMENTATION=0 \
-DCMAKE_SHARED_LINKER_FLAGS="-L$ROOTFS/usr/lib/arm-linux-gnueabihf -L$ROOTFS/lib/arm-linux-gnueabihf" \
-DCMAKE_C_FLAGS="-I$ROOTFS/usr/include -O2 -pipe -march=armv6 -mfpu=vfp -mfloat-abi=hard -I$ROOTFS/usr/include/arm-linux-gnueabihf -I$ROOTFS/usr/local/include" \
-DCMAKE_SYSROOT=/media/michael/rootfs ../
  • 注意,-DCMAKE_SYSROOT=/media/michael/rootfs在交叉编译中很重要,会无法正确链接.

0x5 内网转发

SSH转发

  • 如果自己有公网的IP或者VPS,可以通SSH远程转发的方式,把本地HLS流服务暴露出去.这里要确保VPS里的sshd_configGatewayPorts=yes这一行,不然SSH服务器只能绑定localhost:8080.下面命令成功后,可以通过http://<vps-domain>:8080访问里内网里的HLS服务了.
1
~$ sudo ssh -NfC -R 8080:localhost:80 user@<vps-domain>

ngrok转发

  • ngrok是一个内网穿透服务,需要注册才能使用,免费版帐号只能运行一个且有功能限制的实例.它在github的源码已经不更新,不过自己可以使用该源码搭建自己的私服.

  • 它的使用也相当的容易,无须配置.先注册子一个帐号,再下载一个对应平台的ngrok客户端程序.在须要转发的内网机内,添加帐号里的Authtoken.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# 添加自己帐号里的authtoken
~$ ./ngrok authtoken Nqabz7VHLTEoQEqhXxpK_o8yqiqE9DVyB171LZez9

# 把本地80端口的服务转发去.
~$ ngrok http 80
ngrok by @inconshreveable (Ctrl+C to quit)
Session Status online
Account yjdwbj (Plan: Free)
Version 2.3.35
Region United States (us)
Web Interface http://127.0.0.1:4040
Forwarding http://7ce99246.ngrok.io -> http://localhost:80
Forwarding https://7ce99246.ngrok.io -> http://localhost:80
Connections ttl opn rt1 rt5 p50 p90
0 0 0.00 0.00 0.00 0.00

IPv6-PD(Prefix Delegation)模式,动态域名直连

  • 这里就使用dynv6.com的免费域名.注册帐号->激活帐号->创建域名.会创建一个WAN_6的接口,同时会从ISP获得一个IPv6公网地址与IPv6-PD的前缀.

  • 如果是用NAT6的模式,需要在OpenWRT编译源码时,需选择上iptables-nat6,转发DNAT与IPv4下的NAT是一样的原理.确保安装了dhcpcd5 - DHCPv4, IPv6RA and DHCPv6 client with IPv4LL support,安装完成dhcpcd5后启动systemctl start dhcpcd5.

  • 或者使用下面脚本,如:请求wlan0的IPv6地址.

1
2
3
4
5
6
7
8
~# cat > /etc/network/if-up.d/dhcpcd <<EOF
#!/bin/bash
# filename: wlan0-up
if [ "$IFACE" = wlan0 ]; then
dhcpcd wlan0
fi
EOF
~# chmod +x /etc/network/if-up.d/dhcpcd
  • 或者在/etc/network/interfaces里加入post-up.
1
2
3
4
5
6
7
8
9
10
auto  wlan0
iface wlan0 inet static
address 192.168.1.100
netmask 255.255.255.0
network 192.168.1.0
gateway 192.168.1.1
# here is ovs-vsctl add-port ovsbr0 wlan0
pre-up wpa_supplicant -B -Dnl80211 -iwlan0 -c/etc/wpa_supplicant/wpa_supplicant.conf
post-up dhcpcd wlan0
wait-delay 15
  • 查看IPv6路由.
1
2
3
4
5
6
7
8
9
10
~$ ip -6 route
::1 dev lo proto kernel metric 256 pref medium
# 2409:8a55:2412:6b30::/60 就是路由器通过PPPOE获取到的PD
2409:8a55:2412:6b30::/64 dev wlan0 proto ra metric 303 mtu 1492 pref medium
fe80::/64 dev ovs-system proto kernel metric 256 pref medium
fe80::/64 dev ovsbr0 proto kernel metric 256 pref medium
fe80::/64 dev wlan0 proto kernel metric 256 pref medium
fe80::/64 dev wlan8 proto kernel metric 256 pref medium
default via fe80::2276:xxxx:xxxx:ac08 dev wlan0 proto ra metric 303 mtu 1492 pref medium

更新IPv6的脚本

1
2
3
4
5
6
7
8
9
# 在我的实际电脑中,一块网块除了fd开头的内网地址,还有另外四个,但是从外网能ping通的地址只有一个,就是与路由里 IPv6-PD前缀一致的才可以ping通.
if cat /sys/class/net/eth0/operstate | grep -q "up"; then
IPV6=$(ip -6 addr list scope global eth0 | grep -v " fd" | sed -n 's/.*inet6 \([0-9a-f:]\+\).*/\1/p' | head -n 1)/128
else
IPV6=$(ip -6 addr list scope global wlan0 | grep -v " fd" | sed -n 's/.*inet6 \([0-9a-f:]\+\).*/\1/p' | head -n 1)/128
fi

TOKEN=<dynv6.com Http Token >
curl -fsS -o myddns_ipv6.dat --stderr myddns_ipv6.err "http://dynv6.com/api/update?hostname=<域名全称>&token=${TOKEN}&ipv6=${IPV6}"
  • Luci--> Network--> Firewall--> Traffic Rules里添加一条规则,通过IPv6地址访问它的80端口,在配置文件里显示如下:
1
2
3
4
5
6
7
8
9
10
root@OpenWrt:~# cat /etc/config/firewall
[....]
config rule
option src 'wan'
option name 'camera'
option target 'ACCEPT'
option family 'ipv6'
option dest 'lan'
list proto 'tcp'
option dest_port '80'
  • 如果使用NAT6,且ip6tables -P INPUT ACCEPT话,默认公网就可以通过路由器的IPv6地址,直接访问到路由的80端口,所以这里把nginx服务改成listen [::]:8888 default_server;.

  • 因为是公网可以了访问,可以在系统里安装防火墙脚本apt-get install ufw.

  • 定时刷新DDNS的地址,并且定时刷新OpenWrt里的防火墙规则,否则无法转发端口到内网访问。

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    ~$ cat update.sh
    ETH=eth0
    if cat /sys/class/net/${ETH}/operstate | grep -q "up"; then
    IPV6=$(ip -6 addr list scope global ${ETH} | grep -v " fd" | sed -n 's/.*inet6 \([0-9a-f:]\+\).*/\1/p' | head -n 1)
    else
    IPV6=$(ip -6 addr list scope global wlan0 | grep -v " fd" | sed -n 's/.*inet6 \([0-9a-f:]\+\).*/\1/p' | head -n 1)
    fi

    echo "ipv6 addr $IPV6"
    TOKEN=<dynv6.com token>
    DOMAIN=yjdwbj.dynv6.net
    curl -fsS -o myddns_ipv6.dat --stderr myddns_ipv6.err "http://dynv6.com/api/update?hostname=${DOMAIN}&token=${TOKEN}&ipv6=${IPV6}"
    #IPV4=$(ip addr show ${ETH} | sed -rn '/[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}/p' | awk '{print substr($2,1,length($2)-3)}')
    IPV4=$(ifconfig | sed -En 's/127.0.0.1//;s/.*inet (addr:)?(([0-9]*\.){3}[0-9]*).*/\2/p')
    RULE_ID=$(ssh route uci show firewall | grep "2122" | awk -F'.dest_port=' '{print $1}')
    CMD=$(echo ssh route "uci set ${RULE_ID}.dest_ip='${IPV6}'; uci commit firewall")
    $CMD
    V2RAY_ID=$(ssh route uci show firewall | grep "v2ray" | awk -F'.name=' '{print $1}')
    CMD=$(echo ssh route "uci set ${V2RAY_ID}.dest_ip='${IPV6}'; uci commit firewall; /etc/init.d/firewall restart")
    $CMD

golang交叉编译

  • 使用golang交叉编译v2ray-core.可以运行一个v2ray-core的服务提供中继服务。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
 if [ -d v2ray-core ]; then
cd v2ray-core
git pull
else
git clone https://github.com/v2fly/v2ray-core.git
cd v2ray-core
fi
DOWNLOAD="curl -s -L -x socks5://127.0.0.1:3080 -o "
GOMOD_VER=$(grep '^go\ ' go.mod | awk '{print $2}')
LAST_GO=$(ls -l /usr/bin/go | awk '{print $NF}')
if [ -z ${GOMOD_VER} ]; then
sudo apt-get install golang
else
sudo apt-get install golang-${GOMOD_VER} golang-${GOMOD_VER}-go
sudo ln -svf /usr/lib/go-${GOMOD_VER}/bin/go /usr/bin/go
fi

GOOS=linux
#GOARCH=amd64
VERSIONTAG=$(git describe --abbrev=0 --tags)
CODENAME="user"
LDFLAGS="-s -w -buildid= -X v2ray.codename=${CODENAME} -X v2ray.build=$(date '+%Y%m%d-%H%M%S') -X v2ray.version=${VERSIONTAG}"
GO111MODULE=on
TMP=$(mktemp -d)
GOPROXY=https://goproxy.cn
go mod download
env CGO_ENABLED=0 GOARCH=arm GOARM=6 go build -o "$TMP"/v2ray"${EXESUFFIX}" -ldflags "$LDFLAGS" ./main
${DOWNLOAD} "$TMP"/geoip.dat "https://github.com/v2fly/geoip/raw/release/geoip.dat"

echo ">>> Download latest geosite..."
${DOWNLOAD} "$TMP"/geosite.dat "https://github.com/v2fly/domain-list-community/raw/release/dlc.dat"
rm -rf build
mv ${TMP} build
sudo ln -svf ${LAST_GO} /usr/bin/go

基于Docker的多架构交叉编译


谢谢支持