注:本文分析涉及到的源码基于Django stable/2.0.x 分支。
计算机大部分思想都是来自于现实生活,所以完全可以用日常生活积累的常识去理解计算机里面的概念。
比如开发一套完整的软件系统,就好比去开发一片商业区。开发商业区的第一步就是要有一块空地;软件系统也是,第一步需要有一个空的项目。在Django世界里,这很简单,用下面命令就可以创建Django空项目。
django-admin.py startproject demo_project
有了一块空地后,下一步就是在它上面建一栋栋大楼(以及给大楼起名)。而在Django中,与之对应的就是创建app的过程,同样非常简单的执行下面命令就可,其中demo_app1就是给大楼起的名字。
python manage.py startapp demo_app1
然而要弄好一片商业区必然没这么简单,还需要对功能不同的大楼进行不同的设计,建造和装修,以及道路指引牌,甚至还有和安全相关的一系列配套设施等等。
这些在后续的文章中会一一道来,这篇文章并不打算继续介绍它们,而是先深入分析上面两个看似非常简单的过程(startproject和startapp),因为本文并不想写成如何教人使用Django的说明书类的文章。
目前为止仅仅执行两个命令,就可以让Django帮你创建好了项目。下文将拨开云雾,来分析它们背后实际的代码逻辑。
好了,先晒源码,
django-admin.py
源码:
#!/usr/bin/env python
from django.core import management
if __name__ == "__main__":
management.execute_from_command_line()
manage.py
源码:
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "demo_project.settings")
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv)
文件django-admin.py
和manage.py
都用到了django.core.management
模块,并执行execute_from_command_line
方法。
此外后者多做了两点:1)设置环境变量DJANGO_SETTINGS_MODULE
为当前项目的settings文件;2)判断Django是否能被正常导入使用。
上面的第一点不是本文的重点,以后的文章再来讨论(注意这里不讨论,并不是说DJANGO_SETTINGS_MODULE
不重要,恰恰相反,它很重要,它是Django项目的入口,指向项目的配置文件,该配置文件中指向ROOT_URLCONF
,ROOT_URLCONF
指向视图以及其他的部分,Django项目需要定位到它们之后才能正常运行)。
继续看django.core.management.execute_from_command_line
方法:
def execute_from_command_line(argv=None):
"""Run a ManagementUtility."""
utility = ManagementUtility(argv)
utility.execute()
其中execute_from_command_line
方法只是实例化了类ManagementUtility
,接下来的重点查看ManagementUtility.execute
方法。
该方法有点长,为了方便我们拆开来分析(源码位置django/core/management/__init__.py
):
def execute(self):
"""
Given the command-line arguments, figure out which subcommand is being
run, create a parser appropriate to that command, and run it.
"""
try:
subcommand = self.argv[1]
except IndexError:
subcommand = 'help' # Display help if no arguments were given.
上面这段代码的目的在于获取命令参数self.argv[1]
;如果用户没有输入命令参数,将用help命令为默认的参数,对应本文开头的django-admin.py startproject demo_project
和python manage.py startapp demo_app1
两条指令,系统获取到的命令参数为startproject
和startapp
。
# Preprocess options to extract --settings and --pythonpath.
# These options could affect the commands that are available, so they
# must be processed early.
parser = CommandParser(None, usage="%(prog)s subcommand [options] [args]", add_help=False)
parser.add_argument('--settings')
parser.add_argument('--pythonpath')
parser.add_argument('args', nargs='*') # catch-all
try:
options, args = parser.parse_known_args(self.argv[2:])
handle_default_options(options)
except CommandError:
pass # Ignore any option errors at this point.
接下来用CommandParser
类来解析剩下的命令行参数(该类只是对类ArgumentParser
的简单封装),这段代码主要是预处理settings
和pythonpath
两个可选参数。解析这两个参数到options
中(其他参数放在args
),然后由方法handle_default_options
去处理options
。
def handle_default_options(options):
"""
Include any default options that all commands should accept here
so that ManagementUtility can handle them before searching for
user commands.
"""
if options.settings:
os.environ['DJANGO_SETTINGS_MODULE'] = options.settings
if options.pythonpath:
sys.path.insert(0, options.pythonpath)
从上面代码中可以看到handle_default_options()
根据对象options
的settings
和pythonpath
来设置环境变量和设置python模块的搜索路径。
继续分析ManagementUtility.execute
函数:
try:
settings.INSTALLED_APPS
except ImproperlyConfigured as exc:
self.settings_exception = exc
except ImportError as exc:
self.settings_exception = exc
这段代码可能会让读者诧异,莫名出现了settings
对象,其实在这个源代码文件开头有行代码from django.conf import settings
引入了settings
,在执行这行代码时,会读取os.environ
中的DJANGO_SETTINGS_MODULE
配置,加载项目配置文件后生成了settings
对象(这里先简单的说明下,以后会有专门的文章讲配置文件加载过程)。
代码settings.INSTALLED_APPS
是用来导入配置文件中所有app
(通过查看文件django/conf/__init__.py
,发现settings = LazySettings()
,它具有__getattr__
方法,所以这行代码相当于调用了LazySettings().__getattr__(INSTALLED_APPS)
,该方法获取属性时如果没有导入配置则导入)。
if settings.configured:
# Start the auto-reloading dev server even if the code is broken.
# The hardcoded condition is a code smell but we can't rely on a
# flag on the command class because we haven't located it yet.
if subcommand == 'runserver' and '--noreload' not in self.argv:
try:
autoreload.check_errors(django.setup)()
except Exception:
# The exception will be raised later in the child process
# started by the autoreloader. Pretend it didn't happen by
# loading an empty list of applications.
apps.all_models = defaultdict(OrderedDict)
apps.app_configs = OrderedDict()
apps.apps_ready = apps.models_ready = apps.ready = True
# Remove options not compatible with the built-in runserver
# (e.g. options for the contrib.staticfiles' runserver).
# Changes here require manually testing as described in
# #27522.
_parser = self.fetch_command('runserver').create_parser('django', 'runserver')
_options, _args = _parser.parse_known_args(self.argv[2:])
for _arg in _args:
self.argv.remove(_arg)
# In all other cases, django.setup() is required to succeed.
else:
django.setup()
第一行代码表示如果settings
对象已经配置好,就会执行django.setup()
方法(注:第一个分支专门针对runserver
启动的服务,并且使用autoreload机制的时候做的特殊处理,autoreload机制也是挺有意思的地方,后续也会写专门的文章来说明,这里先简单略过,只需要知道其实第一个分支的autoreload.check_errors(django.setup)()
也是执行了django.setup()
即可)
接着看django.setup()
,在文件django/__init__.py
中:
from django.utils.version import get_version
VERSION = (2, 0, 5, 'alpha', 0)
__version__ = get_version(VERSION)
def setup(set_prefix=True):
"""
Configure the settings (this happens as a side effect of accessing the
first setting), configure logging and populate the app registry.
Set the thread-local urlresolvers script prefix if `set_prefix` is True.
"""
from django.apps import apps
from django.conf import settings
from django.urls import set_script_prefix
from django.utils.log import configure_logging
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
if set_prefix:
set_script_prefix(
'/' if settings.FORCE_SCRIPT_NAME is None else settings.FORCE_SCRIPT_NAME
)
apps.populate(settings.INSTALLED_APPS)
django.setup()
主要做了3件事:1)对logging模块进行了配置处理;2)设置当前线程的script_prefix
,可以理解成当前目录的前缀;3)循环载入之前从配置文件中导入的apps和models:(通过from django.apps import apps
去查看文件django/apps/registry.py
看到apps = Apps(installed_apps=None)
,这里Apps初始化的时候定义了全局的多个字典、变量、线程锁等,然后通过apps.populate(settings.INSTALLED_APPS)
方法循环载入之前从配置文件中导入的app和model,该方法是线程安全和幂等的,但不可重入,也等下篇文章再来详细讲解)。
继续ManagementUtility.execute
函数:
self.autocomplete()
上面这行代码主要为了提供命令自动补全功能。不详细讲了,但是需要知道Bash几个内置的变量,COMP_WORDS
: 类型为数组,存放当前命令行中输入的所有单词;COMP_CWORD
: 类型为整数,当前光标下输入的单词位于COMP_WORDS
数组中的索引;COMPREPLY
: 类型为数组,候选的补全结果。
if subcommand == 'help':
if '--commands' in args:
sys.stdout.write(self.main_help_text(commands_only=True) + '\n')
elif not options.args:
sys.stdout.write(self.main_help_text() + '\n')
else:
self.fetch_command(options.args[0]).print_help(self.prog_name, options.args[0])
# Special-cases: We want 'django-admin --version' and
# 'django-admin --help' to work, for backwards compatibility.
elif subcommand == 'version' or self.argv[1:] == ['--version']:
sys.stdout.write(django.get_version() + '\n')
elif self.argv[1:] in (['--help'], ['-h']):
sys.stdout.write(self.main_help_text() + '\n')
else:
self.fetch_command(subcommand).run_from_argv(self.argv)
这段代码前几个分支,主要是处理help
命令和version
命令,最后一个分支self.fetch_command(subcommand).run_from_argv(self.argv)
才是我们一开始讨论的那两个命令的真实入口。
其中fetch_command
调用get_commands
从下面几个目录找命令对象(命令类都继承自BaseCommand
,并实现handle()
方法)
-
django/core/management/commands
目录 -
<INSTALLED_APPS>/management/commands/
目录
根据返回的subcommand
实例,执行run_from_argv()
方法。
对应我们文章开始讨论的地方,也就是相当于调用了startproject.run_from_argv(self.argv)
和startapp.run_from_argv(self.argv)
,它们对应的源码如下
core/management/commands/startproject.py
from django.core.management.templates import TemplateCommand
from ..utils import get_random_secret_key
class Command(TemplateCommand):
help = (
"Creates a Django project directory structure for the given project "
"name in the current directory or optionally in the given directory."
)
missing_args_message = "You must provide a project name."
def handle(self, **options):
project_name = options.pop('name')
target = options.pop('directory')
# Create a random SECRET_KEY to put it in the main settings.
options['secret_key'] = get_random_secret_key()
super().handle('project', project_name, target, **options)
和core/management/commands/startapp.py
from django.core.management.templates import TemplateCommand
class Command(TemplateCommand):
help = (
"Creates a Django app directory structure for the given app name in "
"the current directory or optionally in the given directory."
)
missing_args_message = "You must provide an application name."
def handle(self, **options):
app_name = options.pop('name')
target = options.pop('directory')
super().handle('app', app_name, target, **options)
这两个命令都继承了TemplateCommand
类,而且它们的代码逻辑也几乎一样,只是传入的参数不同,所以接下来通过分析TemplateCommand.handle()
即可得知真相。
细心的读者可能会发现,上面明明调用的是方法run_from_argv()
,但是startproject
和startapp
两命令都没有这个方法,其实方法run_from_argv()
是它们继承的父类BaseCommand
里的方法(具体可以查看源码django/core/management/base.py
),run_from_argv()
最后会调用它们的handle()
方法。
现在已经距离真相越来越近了,继续回到TemplateCommand.handle()
。
handle
方法代码也有点多,只挑关键的来分析
def handle(self, app_or_project, name, target=None, **options):
...
for root, dirs, files in os.walk(template_dir):
...
for filename in files:
old_path = path.join(root, filename)
new_path = path.join(top_dir, relative_dir,
filename.replace(base_name, name))
for old_suffix, new_suffix in self.rewrite_template_suffixes:
if new_path.endswith(old_suffix):
new_path = new_path[:-len(old_suffix)] + new_suffix
break # Only rewrite once
...
# Only render the Python files, as we don't want to
# accidentally render Django templates files
if new_path.endswith(extensions) or filename in extra_files:
with open(old_path, 'r', encoding='utf-8') as template_file:
content = template_file.read()
template = Engine().from_string(content)
content = template.render(context)
with open(new_path, 'w', encoding='utf-8') as new_file:
new_file.write(content)
else:
shutil.copyfile(old_path, new_path)
if self.verbosity >= 2:
self.stdout.write("Creating %s\n" % new_path)
try:
shutil.copymode(old_path, new_path)
self.make_writeable(new_path)
except OSError:
self.stderr.write(
"Notice: Couldn't set permission bits on %s. You're "
"probably using an uncommon filesystem setup. No "
"problem." % new_path, self.style.NOTICE)
...
核心逻辑是遍历处理template_dir
目录下的文件,这里的template_dir
在startproject
命令和startapp
命令中分别对应目录django/conf/project_template/
和目录django/conf/app_template/
。目录结构如下
为了更好的理解,先把文章一开始执行
startproject
和startapp
命令生成的目录结构也拿出来,对比后发现几乎完全一样,除了文件后缀名不一样:其实到这里为止,已经能猜到大致过程了,在执行
startproject
的时候,会遍历django/conf/project_template/
下面的文件,把源文件的后缀.py-tpl
改成.py
,然后根据设置好的模版引擎生成相应的文件。这样项目就创建完成了。同样,startapp
也是一样的道理。下面几行代码是专门改文件后缀的
for old_suffix, new_suffix in self.rewrite_template_suffixes:
if new_path.endswith(old_suffix):
new_path = new_path[:-len(old_suffix)] + new_suffix
break # Only rewrite once
其中self.rewrite_template_suffixes就是个python 元组对象,里面包含了原后缀名(py-tpl),以及新后缀名(py)
# Rewrite the following suffixes when determining the target filename.
rewrite_template_suffixes = (
# Allow shipping invalid .py files without byte-compilation.
('.py-tpl', '.py'),
)
现在还剩下模版引擎这块,照样先晒代码
# Only render the Python files, as we don't want to
# accidentally render Django templates files
for filename in files:
if new_path.endswith(extensions) or filename in extra_files:
with open(old_path, 'r', encoding='utf-8') as template_file:
content = template_file.read()
template = Engine().from_string(content)
content = template.render(context)
with open(new_path, 'w', encoding='utf-8') as new_file:
new_file.write(content)
else:
shutil.copyfile(old_path, new_path)
这段代码主要是读取模版文件里面的内容,通过调用方法Engine().from_string()
生成Template
模版对象。
比如下面settings.py-tpl
模版文件内容(只截取了部分)被用来生成Template
模版对象
"""
Django settings for {{ project_name }} project.
Generated by 'django-admin startproject' using Django {{ django_version }}.
For more information on this file, see
https://docs.djangoproject.com/en/{{ docs_version }}/topics/settings/
For the full list of settings and their values, see
https://docs.djangoproject.com/en/{{ docs_version }}/ref/settings/
"""
import os
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/{{ docs_version }}/howto/deployment/checklist/
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = '{{ secret_key }}'
...
接着template.render()
会根据context
内容渲染模版,context
内容如下
context = Context({
**options,
base_name: name,
base_directory: top_dir,
camel_case_name: camel_case_value,
'docs_version': get_docs_version(),
'django_version': django.__version__,
}, autoescape=False)
可见,context
包含的docs_version
、django_version
及options
里面的内容会用来替换模版文件里被{{}}
包含的内容,比如{{ docs_version }}
,然后替换后的新内容会被写入到对应的新文件里,这样Django就帮忙生成了项目需要的所有默认文件。
到现在为止,我们已经知道了文章开头那两个简单命令具体做了什么,可以让我们轻松的完成项目的创建。
最后总结:
startproject
和startapp
属于初期创建项目阶段的命令,所以更多的是完成项目配置相关的工作,之后再根据用户输入的参数subcommand
到命令工具集中去找到对应的命令对象。继承自TemplateCommand
类的命令对象,使用模版引擎把相应template_dir
下面的模版文件渲染生成新项目需要的文件和目录,最终Django就帮我们创建好了新项目。