Vhi,fddlmZddlZddlZddlZddlmZmZmZm Z m Z ddl Z ddl Z e rddlmZddlmZddlmZgdZej*eZ d ddZ dddd  dd Z dd dddd  dd ZdddZ ddd ddZ d ddZddZdZy)) annotationsN)AnyIOOptional TYPE_CHECKINGUnion) InputTypeExportedProgram)FileLike)compilelist_mode_options list_optionscudagraph_mark_step_beginc$ddlm}||||S)au Compile a given FX graph with TorchInductor. This allows compiling FX graphs captured without using TorchDynamo. Args: gm: The FX graph to compile. example_inputs: List of tensor inputs. options: Optional dict of config options. See `torch._inductor.config`. Returns: Callable with same behavior as gm but faster.  compile_fxconfig_patchesr)gmexample_inputsoptionsrs H/home/dcms/DCMS/lib/python3.12/site-packages/torch/_inductor/__init__.pyr r s"' b. AA package_pathinductor_configsc6ddlm}ddlm}t ||s t d|j td||tjd|t |tjtfr |jr|jsNt |tt j"fr$t!j$|j'ds Jd ||xsi}d |d <|j)d r td |t*|||S)a; Compiles the exported program with AOTInductor, and packages it into a .pt2 artifact specified by the input package_path. To load the package, you can call ``torch._inductor.aoti_load_package(package_path)``. An example usage is as follows: .. code-block:: python ep = torch.export.export(M(), ...) aoti_file = torch._inductor.aoti_compile_and_package( ep, package_path="my_package.pt2" ) compiled_model = torch._inductor.aoti_load_package("my_package.pt2") To compile and save multiple models into a single ``.pt2`` artifact, you can do the following: .. code-block:: python ep1 = torch.export.export(M1(), ...) aoti_file1 = torch._inductor.aot_compile( ep1, ..., options={"aot_inductor.package": True} ) ep2 = torch.export.export(M2(), ...) aoti_file2 = torch._inductor.aot_compile( ep2, ..., options={"aot_inductor.package": True} ) from torch._inductor.package import package_aoti, load_package package_aoti("my_package.pt2", {"model1": aoti_file1, "model2": aoti_file2}) compiled_model1 = load_package("my_package.pt2", "model1") compiled_model2 = load_package("my_package.pt2", "model2") Args: exported_program: An exported program created through a call from torch.export package_path: Optional specified path to the generated .pt2 artifact. inductor_configs: Optional dictionary of configs to control inductor. Returns: Path to the generated artifact rr r)aot_inductor_minifier_wrapperz!Only ExportedProgram is supportedz[exported_program.example_inputs is required to be set in order for AOTInductor compilation.zYou no longer need to specify args/kwargs to aoti_compile_and_package as we can get this information from exported_program.example_inputs..pt2zVExpect package path to be a file ending in .pt2, is None, or is a buffer. Instead got Tzaot_inductor.packagezaot_inductor.output_pathzoPlease pass in a package path to aot_inductor_compile() instead of setting the aot_inductor.output_path config.r) torch.exportr debugr isinstance ValueErrorr RuntimeErrorlogwarningioIOBaserwritableseekablestrosPathLikefspathendswithget_aoti_compile_and_package_inner)exported_program_deprecated_unused_args_deprecated_unused_kwargsrrr r s raoti_compile_and_packager73s1h-4 & 8<==&&. +  *.G.S  S  |bii_ 5%%'%%' |c2;;%7 8 ,'008  aam`no  (-2/3+,67 >  )'!)  rF) load_and_runcheck_accuracyrrc|r|t|dk(sJdddlm}t|tj j sJ|xsi}t||||}t|tsJ||D cgc]I} tjj| ddk(r"tjj| dK} } t| dk(rT|D cgc]I} tjj| ddk(r"tjj| dK} } | ddz}|||} | |k(sJ|s|r:t|} |r%dd l m} m}|d k(}||| |d || s| d |S| |i||Scc} wcc} w)a: See docstring for aoti_compile_and_package. If `load_and_run` is True, this function will load the compiled model and run it. This is for the minifier to check the correctness of the compiled model. If `check_accuracy` is set, this function will check the accuracy of the compiled model against gm. kwargs must be None if check_accuracy is set. "strict_accuracy" means "we will minify any time we see anything that diverges", whereas "accuracy" is more conservative, and will only minify if there is a meaningful fp64 divergence rzRwhen checking for accuracy, the inputs must have been flattened and kwargs is Noner) package_aotirz.soz.cppr!) AccuracyErrorsame_two_modelsaccuracyT)only_fwd require_fp64 ignore_non_fpzBad accuracy detected)lenpackager;r$torchfx GraphModule aot_compilelistr.pathsplitextaoti_load_packagetorch._dynamo.debug_utilsr=r>)rargskwargsr8r9rrr; aoti_filesfilerJrescompiled_modelr=r>not_strict_accuracys rr3r3s0~V!1 ` 1& b%((.. // / \rFRv7GHJ j$ '' '# ww%a(E1 GG  T "1 %  t9>'77##D)!,6  &q)D Aw' |Z 0C ,  ~*<8  P #1J"> "01 $$;<<  D +F + M s3AE=AFc"ddlm}|||S)a Loads the model from the PT2 package. If multiple models were packaged into the PT2, this will load the default model. To load a specific model, you can directly call the load API .. code-block:: python from torch._inductor.package import load_package compiled_model1 = load_package("my_package.pt2", "model1") compiled_model2 = load_package("my_package.pt2", "model2") Args: path: Path to the .pt2 package run_single_threaded (bool): Whether the model should be run without thread synchronization logic. This is useful to avoid conflicts with CUDAGraphs. r) load_package)run_single_threaded)torch._inductor.packagerV)rJrWrVs rrLrLs(5 2E FFrr<cFddlm}m}|||||\}}||||S)a Ahead-of-time compile a given FX graph with TorchInductor into a shared library. Args: gm: The FX graph to compile. args: Example arguments kwargs: Example keyword arguments options: Optional dict of config options. See `torch._inductor.config`. Returns: Path to the generated shared library, or a list of files generated by AOTI if aot_inductor.package=True. TODO: make it return a list by default r)_aoti_flatten_inputscompile_fx_aotr<r)rrZr[)rrNrOrrZr[flat_example_inputss rrHrHs9*A#7 D&'$    rc iddidddddddd} |r||S|S#t$r4}td|ddj|j|d }~wwxYw) aReturns a dictionary describing the optimizations that each of the available modes passed to `torch.compile()` performs. Args: mode (str, optional): The mode to return the optimizations for. If None, returns optimizations for all modes dynamic (bool, optional): Whether dynamic shape is enabled. Example:: >>> torch._inductor.list_mode_options() triton.cudagraphsT) max_autotunecoordinate_descent_tuning)r_r^r`)defaultzreduce-overheadzmax-autotune-no-cudagraphsz max-autotunezUnrecognized mode=z, should be one of: z, N)KeyErrorr&joinkeys)modedynamic mode_optionses rrr&s   !)-' !!%)- 0L&%)|D!;|;  &:499\EVEVEX;Y:Z [  s A/AAc`ddlm}|j}t|j S)zReturns a dictionary describing the optimizations and debug configurations that are available to `torch.compile()`. The options are documented in `torch._inductor.config`. Example:: >>> torch._inductor.list_options() r)config)torch._inductorrjget_config_copyrIrd)rjcurrent_configs rrrPs)'%+%;%;%=N ##% &&rcddlm}|y)zJIndicates that a new iteration of inference or training is about to begin.rmark_step_beginN)cudagraph_treesrpros rrrbs 0r)N)rtorch.fx.GraphModulerzlist[InputType]rOptional[dict[str, Any]])NN)r4r rzOptional[FileLike]rrsreturnr-)rztorch.nn.ModulerN tuple[Any]rOrsr8boolr9 Optional[str]rz Optional[Union[str, io.BytesIO]]rrs)F)rJr rWrvrtr) rrrrNrurOrsrrsrtzUnion[str, list[str]])rerwrfzOptional[bool]rtzdict[str, Any])rtz list[str]) __future__rr)loggingr.typingrrrrrtorch._inductor.configrEtorch.fxtorch._inductor.utilsr r"r torch.typesr __all__ getLogger__name__r'r r7r3rLrHrrrrrrs"  ::/,$ g! )-BB#B&B0!"e (,15 e%e % e / e eX(, M $(5915MM M % M  M"M3M/M`G8(, )-   % &   F;?' ')7''T'$r